Big blackout. What happened? 40

[Skogsgurra]

When I got words about the big outage, I immediatley went to my puter to find out what my engineering friends in the US had to say about it. But no Eng-Tips page available. Of course I can understand that. No power - no Internet.

Power was restored piece by piece and I now find Eng-Tips up and running again. My question is still valid: What happened?

Glad to see you again!

Gunnar Englund, Sweden

 

[margaret695]

Modula2: I checked out the Hydro Quebec web site and found a lot of information about their transmission to other areas, but haven't found any thing about that deliberately out of phase whatever... as of yet.

Here is a link to their path diagram for metered interchanges:

http://www.transenergie.com/oasis/info/ilot/guide_en.pdf

I will keep on looking to see if I can find some more information.

Margaret

 

[lengould]

Almost certainly Hydro Quebec uses the thyristor DC - AC converters where its transmission lines interconnect with others to limit any effects of customer systems on it's own system. Which may also be why it is not a member of the regional group, which could impose overriding requirements on this(?).

Which is what my previous post was suggesting, each large (state / province sized) area should implement similar load control systems at their borders to enable islanding in the next event. And, pardon me but I tend to think Ontario with 30 Gw generation should be able to operate as an island for at least a little time. Surely they're no all induction generators, and controlled emergency load shedding for peaks should be able to handle. No?

 

[margaret695]

Whoops.... got my links that I had copied confused. Here is the path diagram.

http://www.transenergie.com/oasis/hqt/en/schemas.htmlx

Margaret

 

[margaret695]

lengould: I just opened some of the individual diagrams for Hydro Quebecs schematic.

You are correct, they are using DC to AC convertors, etc.

I knew that that was how the lines from up North operate... because of the very long distances involved... and now have found out that that is what Hydro Quebec does for selling power to other networks.

Margaret

 

[jbartos]

Suggestion: Assuming that the generators will be sensitively protected and trip on a grid condition that is potentially harmful to them, then it is necessary to concentrate on the grid and transmission line to minimize or eliminate outages.
For example, consider 500kV Transmission Tine (TL) that can carry about 1000A. This gives kVA=sqrt3 x 500kV x 1000A = 866,025kVA
If this type of TL malfunction, the grid will be somewhat affected. Now, if by some coincidence more transmission of this size trip, the grid is in trouble. To minimize or eliminate such trouble, the following is needed:
1. Transmission Line Loading. The large transmission lines will either have to have a redundant transmission line on standby or carry a small percentage of MVA such that, if it happens to malfunction and another similar one at the same time (approximately), there will be a loss of power flow tolerable by the grid.
2. Transmission Redundancy. As mentioned in 1. the powerful transmissions need redundancy to be able to switch to another one in case of malfunction in the transmission or in the switchyard.
3. Transmission Topology. It is known from the computer networks that there are Ethernet, Star, Token Ring, etc. topologies. Each one has some advantages and disadvantages.
Therefore, the transmission line topologies should be built as required. E.g. the switchyard should have a topology corresponding to its importance in the grid, e.g. dual ring bus, if needs to be (Why do computer networks or fiber optics backbone have dual buses, e.g. dual token ring?).

Certainly, the transmission lines importance, as evidenced by the blackout, does not lend itself for any kind of monetary speculations, "milking", "Wall Street Greed", etc.
All applicable and proven designs, engineering, and scientific theories have a plenty of room in the electrical power transmission and distribution industry.

 

[margaret695]

lengould: I just spent a while searching on the internet to see if I could find information about Ontario's generating capacity.... with the result that I will freely admit that I am confused. Some web sites put forward the notion that Ontario has enough generating capacity... while others comment on how many of the generating facilities are out of date and that Pickering has apparently had 4 reactors down for years, etc., etc.

As for the "deregulation" situation in Ontario... it seems that the costs per kwh are now above what the consumers are paying and that the Ontario government is making up the difference with tax payers money. Sigh.

From what I have read it seems to me that every different electrical utility, or whatever... whether government or privately owned, in North America seems to have it's own individual attitudes towards how money is allocated for engineering tasks and maintenance and/or upgrading, as well as operation of their networks. Hydro Quebec is government owned... but makes money by selling power to other utilities... hence it seems logical that they have invested a certain amount of money in the physical equipment and transmission lines, etc., that are utilized to sell power.

But as was very well documented a few years back.... Hydro Quebec was.... and may perhaps still be.... some what short of redundant feeders in the area south of Montreal. Okay.... so I know people who were without power for over a month in 98.... during the winter.

In my opinion, this whatever ties in with jbartos mentioned about "milking" and "Wall Street"... and I will include also "Bay Street" greed. And please note that I have the feeling that even government owned utilities may be affected by greed when it comes to certain types of decisions. Sigh.

I think that there are many possible engineering and technical type solutions out there.... to hopefully avoid similar large black outs in the future. But the whole operative whatever is who exactly pays for the improvements. In my opinion, it probably will fall on the consumers. Sigh.

Margaret

 

[lengould]

Margaret695 - Easy to see why much confusion re. Ont. generating capy. e.g. OPG site states they own 24,278 MW of generation in Ontario, but the detailed list includes only 4 of Ont's Nukes. (I know at least the 8 at Bruce (4 working, 2? being re-commissioned, 2 dedicated to heavy water production now mothballed) are owned by Trans Canada Pipelines, bought from British company last year who bought them from Ont Hydro about 2000). TCPL also owns all the big Nat. Gas Turbos like the ones at Pearson Airport, which I think I remember the Rolls Royce engineers at the building I used to work in telling me were 100+ MW each. Not sure who owns the remaining Nukes. If OPG only 4, who owns Pickering 8? or Darlington 4?

http://www.hydroone.com/en/about/quick_facts/

Hydro One - "Transmission Serves Ontario Generators: 174 (includes 78 generating stations owned by Ontario Power Generation)."

I've seen their trading website, listing all the gens in Ont. at about 30,000+ MW. (Fairly evenly divided 1/3 Hydro, 1/3 Nuke, 1/3 Coal+Gas+Wood Waste from Northern Pulp mills). Website seems to be shut down now, least I couldn't find it. Also states "All-time System Peak: 25,629 MW". Balance of 5,000+ must be held in reserve / offline for maintenance etc.

Re. Why Ont. now short of power, my impression is, short term problem due to a safety system which dumps gadolinium into the reactor core of the nukes at Darlington under certain conditions, which apparently happened. Takes about a week for the gadolinium to (decay/get filtered out). Also about (some number between 6 to 12) workable reactors were mothballed in the 1990's, I think largely for political reasons while the NDP govt. was in. Eves PC govt. has been trying to get (some of) them back on line (4 at Darlington), but is taking a lot longer than estimated. Was stupid to shut them down with no alternate strategy, but so goes politics. Hillarious to hear current NDP opposition now on TV criticising PC's for not arranging enough power.

Ah well. Voters have such short memories.

 

[gordonl]

Ontario is a net importer during peak times, which they attempt to hide by giving average import/export values, with of course the vast majority of exports occuring off peak. On any given week imports of 2000MW are common, which comes almost mostly from the ties in the east to New York. The govt agency Ontario Power Generation owns all nukes except Darlington, the PC govt has been pushing privatization of generation and has sold Darlington and approx 400MW of Hydro and will likely keep going if buyers appear. The transmission system which is almost totally owned by another govt agency Hydro One was under threat of privatization but the PC govt recently backed off. With the nukes coming back online (which is required for long term planning) we should be self sufficient, but with rising demand and next to zero development (public or private) we seem to be in trouble even with the nukes online.

It boggles my mind how the privatization can be viewed as positive, I don't see how improved efficiency in these organizations will out weigh the profits taken, not to mention reliability. With deregulation here power prices increased, and the increased profits to the govt agency OPG are being used to offset increased costs, but the private gens stick it in their pockets. So where does the tax payer gain with more privatization? It seems the govt is purely interested in the one time booster shot gained by the asset disposition. I think I'm crossing the line to rant, so end here.

 

[SidiropoulosM]

For some new info on the blackout visit:

http://www.energycentral.com/sections/news/nw_article.cfm?id=4087012

From this article it seems that a large generator in Ohio was lost, which should have prompted operators to cut load. They chose to rely on interies instead, which soon overloaded and began separating the systems. Which, in turn, caused other interties to overload, causing more separation etc.

 

[GRPatel]

As I could understand from the postings, the episode started with the inadequate transmission capacity. There was a very interesting article in IEEE, Spectrum in July 2001 highlighting the defeciency in transmission system. I reproduce the abstract here under.


Energy woes [US National Energy Policy]
Sweet, W. Bretz, E.A.
This paper appears in: Spectrum, IEEE

Publication Date: Jul 2001
On page(s): 48-53
Volume: 38, Issue: 7
ISSN: 0018-9235
References Cited: 0
CODEN: IEESAM
INSPEC Accession Number: 6988066



--------------------------------------------------------------------------------
Abstract:
The long-awaited and much-discounted National Energy Policy report, issued by a task force headed by Vice president Dick Cheney, was not the one-dimensional document critics and adversaries of the Bush administration eagerly awaited. Much of the criticism since release of the report would seem, in fact, to reflect more what people expected to read than what is actually in the report. This paper excerpts the report. In addition, for a contrasting view, the paper excerpts a report prepared by researchers at several national laboratories which presents quite a different outlook on the role fossil fuels need play in the country's energy future. In the report, the very real crisis in electricity has been bundled with concerns about increased prices for gasoline and home heating oil-increases that may prove quite transitory-to sell the public hard on an all-out effort at new production, with environmental protection given much less emphasis. What the administration seeks from the public, at bottom, is much more production of fossil fuels and streamlined or less onerous procedures for approving new plants and infrastructure

You may visit the site:

http://ieeexplore.ieee.org/iel5/6/2...t=48&ared=53&arAuthor=Sweet,+W.;+Bretz,+E.A.;

The article states:
Energy used more economically but Electricity infrastructure deteriorates. Consumers are spending less of their household income on energy now then 1973 energy crunch.But transmission owners have spent less on infrastructure despite growing demand.

It further states:
Since 1989, electricity sales to consumers have increased by 2.1 percent annually, yet transmission capacity has increased increased by 0.8 percent annually.

Transmission projects always get lower investment priorities than the generation projects.

One of the respondent has rightly suggested islanding of the local networks. There was a collapse of large part of national grid last year in India. Some utilities, who had installed islanding relays could survive the unwanted trippings. They also helped to revive the system by feeding necessary power.

 

[SidiropoulosM]

GRPatel, the same authors (Sweet, Betz etc.) have an interesting perspective on the 2003 blackout in a current special issue of Spectrum:

http://www.spectrum.ieee.org/WEBONLY/special/aug03/black03.html

 

[lengould]

gordonl:

You stated OPG owns all nukes except Darlington. If true then they must be leasing etc. the Bruce facility to the (British Energy --> TCPL) etc? group who are operating. Seems more likely Ont. Govt. would be leasing not OPG.

http://www.brucepower.com/brucepower/

"Duncan Hawthorne is President and Chief Executive Officer of Bruce Power. Prior to the recent change in ownership of Bruce Power, Duncan also served as British Energy's Executive Director - North America. He remains a member of the British Energy Board of Directors, "

further -
"Bruce Power employs more than 3,000 highly-skilled employees with experience in the safe and reliable operation of nuclear generating stations. With safety as its first priority, Bruce Power generates enough clean electricity to supply approximately 15 per cent of Ontario's power needs."

Agreed Ont. is net importer at peaks, but IMHO for stupid reasons. According to following, 2 of the 4 BruceA units ars back "summer 2003"? Doesn't say now or later. These guys (politicians) should ALWAYS keep Ont. in net export condition, no reason not to with all the "spare" nukes about.

"Plans for the restart of Bruce A Units 3 and 4 are now in full swing, with both units expected to be online by summer 2003, subject to a thorough environmental assessment and receipt of all necessary regulatory approvals. The restart of two of the Bruce A units, together with performance improvements to Bruce B will deliver another 2000 megawatts of clean electricity into the Ontario market"

 

[jstickley]

WOW. Interesting perspective is right! Just read that IEEE Spectrum article, and can't help but wonder if the author hasn't penned a lot of words he'll have to eat later. There's an awful lot of finger-pointing and sweeping generalities if you ask me. Although most of the issues identified are very much real, the conclusions and overall negative tone of the article are quite bothersome. I'll admit, I believe the electric power industry has some problems, but to basically trash everyone involved from system operators to elected legislators is a little too much.

I don't know about anyone else here, but the allegation that cuts deepest to me is the assertion that the "best students are not going into power engineering". That one gets personal, huh? Nice to know that, in general, power engineers apparently aren't much brighter than trained monkeys, especially since the industry is so simple to understand and its engineering tasks so straightforward.

(BTW, if you're not catching the sarcasm here, maybe he's right!)

 

[jstickley]

Just got this week's issue of Platt's Electric Utility Week. They have extensive coverage regarding the blackout.

See

http://platts.com/features/poweroutage/index.shtml

for more.

 

[gordonl]

lengould,

Yes, for whatever reason Darlington was on my mind, but indeed it is Bruce which was sold not Darlington.

 

[lengould]

jstickley:

I agree with you totally on the inappropriatness of such comments as follows in the Spectrum article.

<quote says Roytelman, echoing a viewpoint that is ubiquitous among the leading U.S. experts on the grid system, &quot;the best students are not going into power engineering.&quot; >

What I don't get (and I suspect is a mere reflection of the quality of the authors), they spend nearly the entire article explaining why all the problems of the grid system are financial and political, then close with the kick at engineers. They havn't indicated a SINGLE REASON why better engineering might resolve the problem, even though several posters here have, in ways far more acceptable.

Is the lack of new grid capacity caused by lack of engineers? How long might it take to assemble a team of engineers who could &quot;great circle&quot; the continent with 10 or so MVA of 1MV DC if they didn't have to pay from their own pockets and fight local wars at every property line?

Very poor reporting job. &quot;leading U.S. experts on the grid&quot;? Ha. He forgot to insert financial there.

 

[SidiropoulosM]

Sometimes we shoot the messenger for the bad message. Ilya Roytelman, a distinguished researcher with many contributions in the field, said that &quot;the best students are not going into power engineering&quot;. Now, what's wrong with that statement? If it's not totally and absolutely correct, it's partially correct, it has an element of truth. Certainly in Roytelman's experience, as he is an academic. And, I can say, in my own experience also. When I graduated in 1974, the brightest students went into digital electronics and Bio-med. In any event, if there is a problem it will not be solved by those who are offended but by those who can face the truth and deal with it.

On the other hand, many bright people have gone into the various fields of electric power and the quality of technical research in our field is the best testamemt to that. Many &quot;average&quot; students have developed into formidable experts when faced by the enormous complexity of the electricity supply systems.

Now, on the subject of the Spectrum paper that I posted above. I really don't know why some have come down on it so hard. Is it because it says that:

&quot;...Long before power had been restored to businesses and residences from New York City to Cleveland, Detroit, and Toronto, politicians and commentators on both sides of the border were pointing fingers...&quot;

&quot;...For more than five years, NERC has sought and failed to get legislative authority to make its rules mandatory...&quot;

&quot;...as growth in demand for electricity has outstripped additions to transmission capacity by a factor of two, the grid itself has come to be ever more thinly stretched...&quot;

...&quot;We all knew something like this was coming along,&quot; a leader in a 1999 DOE study of the transmission grid told IEEE Spectrum, on condition of anonymity. &quot;We were all just waiting for the big one.&quot;...

&quot;...The temptation, at first blush, may be to put the blame squarely on FirstEnergy, the Ohio utility whose operators and managers seem to have been sleeping at the switch. Could this be a mere case of one poorly run company bringing down an entire system—of one rotten apple corrupting the whole barrel?...&quot;

...&quot;Reporters for The Wall Street Journal, The New York Times, and ABC-TV's &quot;Nightline&quot; news analysis show promptly uncovered a pattern of chronic difficulties FirstEnergy had had with regulators. These included penalties for violation of health, safety, and environmental rules, a requirement by auditors that it restate its profits over the last few years, and the cost of replacing electricity from the shut-down Davis-Besse plant, which ran into hundreds of millions of dollars. The company also is saddled with billions of dollars in debt, much of it associated with its acquisition in 2001 of the utility GPU Inc. (Morristown, N.J.), the former owner and operator of the Three Mile Island nuclear plant that suffered a near meltdown in 1979...&quot;

&quot;...Even if the tripped lines and FirstEnergy’s failure to detect and report them prove to be the exclusive initiating cause of the 2003 blackout, it still will be necessary to account for the failure of local and neighboring control authorities to prevent the cascading outages that ensued. As events unfolded, observers were struck at how unevenly various regulating organizations performed...&quot;

&quot;...Long before most states had even begun to set up ISOs, however, it was becoming apparent to federal regulators that the supposedly independent organizations were held hostage by local utilities and other special interests. What’s more, ISOs lacked adequate authority to get transmission lines built and transmission services properly priced...&quot;

&quot;...Six months ago, NERC singled out the Middle West as the one part of the country at risk of a devastating grid breakdown, and put local utilities and regulators that they had to be on high alert this summer. Evidently the message did not get through. Why is that?...&quot;

...&quot;The market forces that caused this,&quot; the report went on to say, &quot;pervade all of North America. Similar effects should be expected in other regions as well,&quot; including the eastern interconnection, where the same kind of migration of technical support from utilities to emergent ISOs, with attendant loss of memory, was apparent....

&quot;...William Hogan of Harvard University’s John F. Kennedy School of Government (Cambridge, Mass.) noted that the Senate Commerce Committee had recently voted to bar FERC from implementing RTO market designs based on the PJM Interconnect model. But, he wrote hopefully, &quot;the blackout should change the game.&quot;...

I can go on and on but I'll end up reprinting the whole article. Open your eyes and read it once again (or more than once if necessary).

 

[jstickley]

Here's a VERY informative analysis of the 8/14/03 system collapse, by International Transmission Company:

http://www.itctransco.com/documents/B_8-14-2003_ITC_Analysis.pdf

No conclusions are drawn, just a statement of available facts from their information and analysis.

 

[lengould]

Sid: I can't find a single instance in your selections which would indicate <poor quality engineering> might be a cause or even a contributer to the problem. Some might point to some operaters asleep at a console, but engineering? It's a tribute to the engineering that the system can survive at all given the management problems described. The <last kick> at engineers was unjustified.


&quot;the supposedly independent organizations were held hostage by local utilities and other special interests. WhatÂ’s more, ISOs lacked adequate authority to get transmission lines built and transmission services properly priced...&quot;

...&quot;The market forces that caused this,&quot;

 

[jstickley]

SidiropoulosM --

I certainly was not shooting the messenger. No personal attack was made (or intended) upon Ilya, nor upon the author. My comments were simply expressing my reaction to the article -- I felt that although it made some good points and summarized some of the main issues, it had an overall tone of negativity which detracted from it's factual content. I also felt that the implied assertion that the blackout was somehow related to a lack of bright students studying the field of power engineering was ridiculous -- while it's true that power engineering programs suffer from a lack of prestige and do have smaller enrollments than other electrical engineering disciplines, I find it difficult to believe that they do not produce quality engineers who are capable of planning and operating a reliable power system.

I stand by my comments (which I'll be happy to elaborate upon more, if need be), and am sincerely sorry if you did not catch my poor attempt at humor.