Flood Mitigation in Coastal Urban Communities using Real-time Stormwater Infrastructure Control and Reinforcement Learning

The traditional gravity-driven stormwater systems that coastal urban communities typically rely on to manage flooding are increasingly stressed by sea level rise and climate change. While recent research has shown that retrofitting these passive systems as smart cyber-physical systems controlled in real-time can improve their performance, methods for automating and optimizing that real-time control is an active area of study. This research explores deep reinforcement learning (RL) to create control policies for these systems. In RL, an agent learns control policies by interacting with an environment and receiving rewards or penalties based on the agent's actions. In this case, an RL agent controls valves in a simulated stormwater system, which uses observed rainfall and tide data as input, and is penalized if flooding occurs. The RL agent must learn to manage the depth of water in two retention ponds by opening and closing valves, which is complicated by the fact that the agent must learn to time releases of water from the ponds with changing tidal conditions at the stormwater system outlet. The RL agent’s performance is compared to (i) a passive, gravity-driven system and (ii) a model predictive control (MPC) strategy. Results show that the RL agent can learn to proactively manage water levels in the retention ponds based on current and forecast conditions, and can reduce flooding compared to the passive, gravity-driven system. In contrast to MPC, which in this case uses a heuristic approach to perform online optimization, the RL agent is better suited for real-time control because it can be trained offline and used with a lower real-time computational cost than MPC, allowing it to scale to larger systems. This research helps to inform control strategies for smart stormwater systems by allowing them to learn from and adapt to a wide range of conditions.