SCADA-less system 3

[Mbrooke]

What would you think if someone was to seriously pitch a system (80,000MW) entirely without SCADA or remote control? Every protection relay, tap changer control and cap bank control independent without any type of communication- except perhaps a few dry contacts to trigger remote alarms for lockout, low SF6 and relay failure. Switching can only be done at the station.


60 years ago no communications was the norm with this being an incentive:

https://www.engineering.com/Advance...sc=212727627.2.1555193147552&__hsfp=592281695

Obviously you need boots on the ground and things like phase angle regulators probably can't be used- but is this technologically feasible?

 

[Mbrooke]

This:

https://video.wired.com/watch/watch-hackers-take-over-a-ukranian-power-station

In 2015, a cyber attack on distribution utility substations in Ukraine shut off power to over 225,000 utility customers for several hours, and was the first time that a cyber attack was publicly acknowledged to have caused a grid power outage. A second cyber attack in Ukraine in 2016 was reported on an electricity control center in the city of Kiev, shutting down substations which controlled 200 megawatts of capacity. The potential for a similar attack on the U.S. grid was seen as a possibility. Some recent reports about foreign hackers targeting the U.S. electric power system and other critical infrastructure are summarized below. While these intrusions have not been reported as having resulted in significant disruptions, concerns have increased over the potential for these intrusions to result in damaging cyber attacks in the future.

Page 1:

https://fas.org/sgp/crs/homesec/R45312.pdf

 

[itsmoked]

I could easily see no scada for security reasons these days.
However, I can't see why realtime data monitoring wouldn't be involved.

Keith Cress
kcress -

http://www.flaminsystems.com

 

[crshears]

Boots on the ground . . . those were the days . . .

If sda with no ca was used, PARs could still be employed, provided the locations with them were made attended so the controls could be hard-wired.

On a related topic . . .

Nowadays SVC stands for Static Var Compensator, but in the very early 2000s my utility still used what were called System Voice Channels for communication between control rooms at attended sites. These 1960s technology SVCs were like a permanent party line, meaning always available, and used copper "Bell" pairs {Eastern Canada usage], microwave paths, and power line carrier as their media.

To use them, one would select the intended SVC via PBX505 or push-button telephone, lift the receiver and listen to learn if the line was already in use, and if not, punch in a "tone dialling" three-character alphabetic designation for the target station. The tone recognition relays at the destination would detect that sequence and activate a bell or tone generator to signify the presence of an incoming call. Highly vituperative language was heard on these SVCs whenever some inconsiderate dolt punched in an access code without listening first, as anyone else already on the line would receive an unexpected and deafening three-tone blast directly into the ear . . . and the Senior System and Regional Operators did not quickly forget the perpetrators.

The SVC comm path was completely immune to cyber attack.

Those were indeed the days . . .

CR

"As iron sharpens iron, so one person sharpens another." [Proverbs 27:17, NIV]

 

[Mbrooke]

Whats stopping such a system today? Obsolete parts? I'm thinking there has to be a way, any way.

 

[HamburgerHelper]

I think this might be more common than you think. Data diodes, devices that only allow one way communication, are sometimes installed to make sure that a site doesn't have external routable communications. Data can be only exported for certain things and all controls have to happen on site. Nuclear I believe has had to operate like this for awhile. if you don't put in a data diode, you have to keep up to date on firewalls and such to meet CIP requirements. Talk to your CIP expert to get the full details on this.

------------------------------------------------------------------------------------------
If you can't explain it to a six year old, you don't understand it yourself.

 

[Mbrooke]

Will do.


Question- in the old days where there was no real time computer screen power flows, how were power plants dispatched?

 

[crshears]

I can't answer that one - I'm not THAT old!

More seriously, automatic frequency/generation control [AGC] systems need tie line bias, area control error, and other like calculations to function, and it would take some probably-way-beyond-me thought to come up with means as to how these would be done in a non-digital cyber-attack-immune manner.

 

[marks1080]

You almost always need some kind of communication with remote terminals, especially if you're using a half way decent line protection scheme. So going SCADAless doesn't necessarily mean you save on all the communication infrastructure. I am a fan of not installing extra SCADA equipment (eg: use the functionality provided by the IED's - don't go out of the way to install 'other boxes').

 

[bacon4life]

The norm 60 years ago was each powerplant and each major substation was staffed by operators 24x7. Extrapolating from locations I am familiar with, 80,000 MW might take 8000 operators.

Since each location was controlled somewhat indepentantly, the system required a larger overall capacity. The trickiest bit seems like incorporating variable renewable generation into the scheme. The trend in generation control has been to automatically redispatch plants every few seconds to minutes so that the overall cost of generation is minimized.

 

[waross]

I was in a hydro plant back in the mid '70s being given a tour by the operator. All of the generators were at 10% output. The phone rang.
Central dispatch required #2generator to go up to 80% output.
The operator went over to the control board and turned a three position control. Center off, spring return to center.
This opened a hydraulic valve and allowed hydraulic fluid into a large hydraulic cylinder connected to a large bell crank.
As the cylinder extended, it opened the gates and allowed more water though the turbine.
While doing this, the operator was watching the PF meter.
He then went over to another control and increased the field to bring up the pF.
Then he went back to the first control and opened the gates more.
As I remember he went back between the two controls 4 or 5 times until he had the set up to 80% output.

The operator told me that the generators were kept on-line at 10% output all the time. That kept them synchronized and warm, ready to pick up a load quickly when needed. They ran at either 10% or 80% as needed.
Some years later the plant was fully automated and is no unmanned.

I wasn't involved but some years later a grievance report passed through our office.
At a similar plant one of the generators was idle.
Load dispatch phoned and instructed that it be placed on-line.
Dispatch was told that the generator was out of service as the synchro-scope was not working.
After an argument the operator was told definitely to put the generator on-line.
(Another case of management talking when they should be listening.)
The operator ran the set up and closed the breaker.
There was a lot of noise and a lot of damage.

The operator was putting in a grievance protesting the disciplinary action that he received for following a direct order from an incompetent superior.



Bill
--------------------
"Why not the best?"
Jimmy Carter

 

[marks1080]

mbrooke,

Most cyber attacks in this industry will likely come from equipment that's been 'infected' at the manufacturing level, before it's even purchased. The industry, in general, is basically ignoring this right now, though it is the biggest threat.

Just because you have networkable equipment, doesn't mean you have to connect it to the internet. Utilities can deploy their own private networks that would require someone to actually be in a station or controlled site to 'hack' the system. Don't need to throw the baby out with the bathwater Also, the utility I work for basically does this (a private network) yet some folks here are still beyond paranoid some internet hacker is going to infiltrate the system. There's absolutely a security issue with private networks when you have equipment connected that's already been infected as mentioned above.

In my experience, the folks in charge of cyber security issues/policy in power utilities have very little technical experience in either utilities or network infrastructure. This is largely the reason some of the NERC/NPCC cyber security rules are absurd.

 

[Mbrooke]

@marks1080- never knew that was possible. How does that happen if you can give more detail?


@waross: how did the operator know the hydro plant needed to raise its output?

 

[davidbeach]

A few thoughts -

The greatest risk of “attack” remains from the inside. It might be intentional or it might be entirely unintentional.

Without at least the data acquisition part of SCADA it will be easier for things to get farther off the rails faster than as things are today.

Lots of units running at well less than full output and under governor control would maintain a good match between load and generation but completely ignore all sorts of economic dispatch considerations.

 

[FreddyNurk]

Part of the issue is that a lot of equipment is designed for interconnectivity with security either as an afterthought, or not at all. Its not impossible to secure such things, but it does take more effort and consideration of the likely attack vectors. If one also considers that most SCADA or equivalent applications require underlying services and software that may also present security issues, then the whole thing gets more complicated.

At the lower level, protocols like Modbus have absolutely no security inherent at all. So if access to a Modbus communications medium can be compromised, then there's a whole level of things that an attacker could do. If a SCADA server is compromised, then generally it has unrestricted access to all the communications devices and also provides for a whole host of issues. As marks1080 said though, there are ways and means to secure systems, but they do take more effort.

I don't know that it would be effective or sensible to consider not providing a system with SCADA or no interdevice communications as a solution to the security issues. Dispatch visibility, outage management and so on would be much more difficult without access to the data and records provided via communications media.

EDMS Australia

 

[waross]

"waross: how did the operator know the hydro plant needed to raise its output? "
A phone call from the central dispatch office.

"The phone rang.
Central dispatch required #2generator to go up to 80% output."

"Load dispatch phoned and instructed that it be placed on-line."

The load dispatch center would be monitoring the loading on the swing set.

Bill
--------------------
"Why not the best?"
Jimmy Carter

 

[waross]

In our little utility with about 5000 homes, the operators were required to enter in the log every 15 minutes the readings of all the sets online.
Oil pressure,
Coolant temp,
Watts,
KWHrs,
Current,
Voltage.
When the output reached a predetermined point, then another set would be either added or taken offline.
The head plant operator was a mechanical guy. His expertise was supervising the 15,000 hr or 30,000 hr overhaul of the diesel engines.
We had three V-8s and two V-12s.
The head operator was not very electrical.
He would report the load level as how many cylinders were on-line.
8 cylinders,
12 cylinders,
16 cylinders (two V-8s),
20 cylinders (V-8 plus a V-12),
etc.
It worked.
The lights stayed on.
Definitely no SCADA unless you consider the manual Data Acquisition to be part of SCADA. grin

Bill
--------------------
"Why not the best?"
Jimmy Carter

 

[crshears]

Dispatch was told that the generator was out of service as the synchro-scope was not working.

Resolutely refusing to attempt to place that unit in service would seem to me have been a much more defensible position than to try it knowing there was a high degree of probability, like a 95% chance, of closing the synch breaker out of phase. And even if I was terminated for refusing such a direct order, it'd probably be a lot easier to find a similar job elsewhere.

Units I dealt with either had dual synchroscopes or, if just a single one, lamps in parallel with it, so "the synchroscope was not working" is a bit vague . . .

Then again, a loss of generator or line-side potential would have rendered the entire synchronizing scheme unavailable.

That being said, absent further information, it'd be hard to know who to credit with the greatest stupidity . . .

CR

"As iron sharpens iron, so one person sharpens another." [Proverbs 27:17, NIV]

 

[waross]

I agree CR. If we make a movie we can call it "Dumb and Dumber II"
Not enough training on either end of the 'phone line.

Bill
--------------------
"Why not the best?"
Jimmy Carter

 

[marks1080]

mbrooke: Look up StuxNet - anyone claiming any credibility in the 'cyber security' space should be well aware of this. Surprisingly you mention it in a 'cyber security' meeting and the MBAs just stare at you with a dumb look on their face.

davidbeach - an 'attack' by definition, cannot be unintentional. The largest cyber threats do not come from the 'inside.' However, I do believe it is entirely possible that some states send utility workers into other states to gain access to power systems. I doubt it happens too often....

Personally, I think its reasonable to assume most, if not all, electronic, networkable equipment used in the power industry is already compromised before you install it. The only thing you want to make sure is that its not compromised by 'the other side.'

*edited for typo