Board Briefs: Carroll Plant SCADA System Transformation

 

At the recent Massachusetts Water Resources Authority (MWRA) Board of Directors meeting on January 15, 2025, Ethan Wenger, MWRA’s Director of SCADA, Metering and Monitoring, delivered a detailed presentation on the ongoing Supervisory Control and Data Acquisition (SCADA) system upgrades at the John J. Carroll Water Treatment Plant. This vital project represents a significant step forward in ensuring the reliability, security, and efficiency of one of the region’s most critical pieces of water infrastructure.

 

The Carroll Water Treatment Plan and Its Role

 

The John J. Carroll Water Treatment Plant, located in Marlborough, Massachusetts, has been a cornerstone of the region’s water supply since its opening in 2005. With a peak capacity of 405 million gallons per day (mgd), the plant supplies drinking water to nearly three million customers across 47 MetroWest and greater Boston communities. Its advanced treatment processes, which include ozone and ultraviolet light (UV) for primary disinfection and sodium hypochlorite with ammonia for secondary disinfection, are fully automated by the SCADA system. This system also oversees corrosion control processes using soda ash and carbon dioxide, fluoride addition for dental health, and other critical operations. Operating 24/7, SCADA is essential to maintaining water quality and reliable service.

 

SCADA Upgrade Project Overview
By 2021, the SCADA system had reached the end of its useful life, with vendor support for critical components such as backup scanner modules no longer available. As newer programmable logic controllers (PLCs) with enhanced security and reliability became available, it became clear that a full system replacement was necessary to safeguard the plant’s operations against both technical failures and emerging cyber threats.

 

The upgrade project, initially approved in 2018, is designed to replace legacy equipment, including PLCs, computers, and hardware, with state-of-the-art technology. A key feature of the project is the construction of a parallel SCADA system alongside the existing one, allowing the plant to continue operating without interruptions during the transition. Redundant control loops have also been implemented, ensuring that the plant remains operational even if the main SCADA system experiences failures.

 

As of January 2025, the SCADA upgrade project has made significant progress. The new system includes over 25,000 input/output (I/O) points for real-time monitoring and control, 800 new computer display screens for streamlined operator interfaces, and a state-of-the-art control room and server room with modern HVAC and fire suppression systems. Additionally, two independent sub-networks have been installed to ensure resilience against failures or cyberattacks.

 

Progress, Challenges, and Collaboration
The plant is divided into two operational halves known as Train A and Train B. These “trains” refer to independent treatment processes within the facility, each capable of handling a significant portion of the plant’s total capacity. This configuration provides redundancy and flexibility, allowing the plant to continue operating even when one train is offline for maintenance or upgrades. In November 2024, during the plant’s annual maintenance period, control signals for Train B were transitioned to the new SCADA system. This process involved meticulously transferring and testing each signal to ensure operational accuracy. By January 2025, Train B was fully operational on the new system, with plans to transition Train A following the same approach.

 

Replacing a SCADA system in a plant that operates 24/7 is no small feat. Transitioning signals wire by wire is labor-intensive and demands rigorous testing at each stage to prevent operational disruptions. The cutover process requires careful planning and precise execution, as each individual wire has to be transferred and verified in a meticulous, step-by-step manner. For example, transferring a wire controlling a valve requires a team effort: one person must initiate the change from the SCADA system, directing the valve to open a specific percentage, such as 25 percent. Another person then monitors the programmable logic controller (PLC) feedback to confirm the signal is being received correctly. Simultaneously, a team member must physically walk out to the site and verify that the valve is indeed moving as instructed, opening precisely 25 percent. This process is repeated for every signal, and with panels like the OZ7 panel in the ozone treatment process containing 450 signals, it becomes a time-consuming and collaborative effort requiring intense focus and precision.

 

Early contract amendments played a crucial role in shaping the project. Amendment 1 extended the design phase by 184 days in 2020 due to COVID-19-related disruptions, while Amendment 2 in 2021 added funding and time to facilitate the construction of the parallel SCADA system. Subsequent amendments addressed emerging challenges, such as supply chain delays and expanded programming requirements, culminating in the current Amendment 5, which requests $1,323,864 to support training, field testing, and extended construction periods.

 

The success of the SCADA upgrade project is a result of collaboration across MWRA divisions and with external partners. Key contributors include Ethan Wenger, MWRA’s Director of SCADA, Metering and Monitoring, who oversees the technical aspects of the project; Valerie Moran, MWRA’s Director of Waterworks, and John Beckley, MWRA’s Program Manager, Monitor and Control, who provide strategic oversight; and Arcadis, Inc., a global design and consultancy firm leading design, programming, and testing efforts.

 

Looking Ahead

 

The upgrades at the Carroll Water Treatment Plant highlight MWRA’s commitment to public health and environmental stewardship. By investing in cutting-edge technology, the Authority is enhancing the plant’s resilience against potential risks, including cyberattacks and equipment failures. These efforts ensure the delivery of clean, safe drinking water for decades to come.

 

As Train B operates on the new SCADA system, the focus now shifts to completing the transition for Train A. With its advanced features and robust design, the new system positions the Carroll Water Treatment Plant as a leader in water treatment innovation and reliability. Through thoughtful planning, collaboration, and investment, the SCADA system upgrade project serves as a model of infrastructure improvement in service of public health and safety.