A rifle attack on an electrical substation near California’s Silicon Valley in April 2013 led to the development of a new tool for grid operators that will enable them to better detect not only a brutal physical attack but also the slightest hint of a hacker looking for vulnerabilities in these critical links in the grid.
The thousands of substations that are nodes in North America’s electrical grid receive high-voltage energy from transmission lines that originate at power plants and step down that voltage so it can enter local distribution networks to power homes and businesses. Although distributed in nature, grid operators worry that the loss of just a few critical substations could trigger an outage that cascades across a region, potentially crippling a major urban center.
Indeed, in 2014, the Wall Street Journal reported the startling findings in confidential report by the Federal Energy Regulatory Commission (FERC): Thirty substations across the U.S. played an outsized role in grid operations; knocking out nine of them could cause a cascading outage capable of bringing down the nation’s grid.
Investigators thought that the intent to trigger such a cascading event may have been behind a 2013 rifle attack on Pacific Gas & Electric’s Metcalf substation in Coyote, Calif., near San Jose, home to Silicon Valley. During the still-unsolved crime, attackers cut fiber optic cables to the facility, and then shot up 17 transformers, resulting in $15 million in damage. The utility had to to re-route power around the damaged substation until repairs could be made.
A rifle assault means the attacker has to come close enough to blast away at a substation. Perhaps more worrisome to grid operators, however, is the possibility of a cyberattack launched remotely from anywhere on the globe.
Stoking those concerns is the fear that a seminal event in using computer networks to bring down a nation’s infrastructure—the December 2015 assault on Ukraine’s power grid—will happen again. In that attack, hackers switched off 30 substations across three energy distribution companies, disrupting electricity supply to around 230,000 end users for up to six hours.
Against those background events, a team of researchers working at the U.S. Energy Department’s Lawrence Berkeley National Laboratory completed work earlier this year on a project to design and implement a tool they say can detect cyberattacks and physical assaults on power distribution networks.
Their tool, developed after three years of work, uses micro phasor measurement units (μPMUs) to collect information about the physical state of the power distribution grid. Combining that data with SCADA (supervisory control and data acquisition) information provides real-time insights into system performance and alerts grid operators to even a minor disruption.
Marriage Made in a Laboratory
Grid operators look to frequency—60 hertz in North America and 50 Hz in Europe, for example—as a primary indicator of system health. Devices known as synchophasors help operators monitor frequency by measuring both the magnitude and the phase angle of the sine waves found in electricity. Synchophasors are able to provide data orders of magnitude faster than SCADA systems that are in common use across the world’s power networks. If installed at facilities such as substations, synchophasors can pay close attention to frequency and alert operators to a system anomaly that deserves attention.
The threat detection application developed at Berkeley Lab marries safety engineering with computer security, says Sean Peisert, a computer scientist in the lab’s Computational Research Division. He led the research effort, along with collaborators at Arizona State University, synchophasor pioneer Power Standards Laboratory, the Electric Power Research Institute, software vendor OSISoft, and utility partners Riverside Public Utilities and Southern Company.
That marriage is important, says John Matranga, director of Customer Innovation and Academia at OSISoft. “What Sean was able to do was bring forth the idea that data is a critical element in determining the cyber state of the grid.” By comparing hard data from the control system with First Principle physics that describe how the grid should function, grid operators can determine if a suspicious event is underway.
Investigators looking into the 2015 Ukraine attack, for example, learned that one or more intruders had gained access to equipment control functions and were stealthily looking for vulnerabilities. Their intrusion began months before the substation attacks were launched.
That sort of “reconnaissance attack” may have involved making small changes to how equipment operated such as threshold adjustments, says Ciaran Roberts, Senior Scientific Engineering Associate at Berkeley Lab. To help counter such a probing threat, the new detection tool uses machine learning so that a system’s long-term nominal operational mode can be compared against real-time SCADA data. Unexpected behaviors immediately make themselves evident; operating system engineers can quickly involve their information technology counterparts to better understand what is going on, Roberts says.
Grid hardening efforts began after the terror attacks of 11 September 2001, says Bryan Owen, Security Chief at OSISoft. Those efforts began with control centers and power plants, and were guided by rules from FERC and the North American Electric Reliability Corporation, which is responsible for the grid’s overall health. Security rules were extended to substations following the Metcalf attack, Owen says.
Up on the Roof
The Berkeley Lab team is extending its work beyond substations to include distributed generation resources such as rooftop solar panels. The worry is that the thousands of solar panels and their electronic equipment could entice a hacker to gain access to power inverters and disrupt a region’s power grid. Such an intrusion could come from something as simple as a software update pushed out by an equipment supplier.
The possibility of an attack has actually been enhanced by industry and government efforts to develop standards for how solar inverters communicate with the grid.
“It is this standardization that presents a vulnerability,” said Daniel Arnold, a Berkeley Lab researcher in announcing the project.
Berkeley Lab will lead work to develop algorithms that counteract attacks on solar inverters by sending opposite signals to nullify malware—similar to what a noise-canceling headphone does.
The three-year, $2.5 million project began in early March and includes industry partners, the National Rural Electric Cooperative Association, and the Sacramento Municipal Utility District.