76 Comments
User's avatar
Meredith Angwin's avatar

An excellent blow-by-blow account of the day of the Spanish blackouts. From a VERY knowledgeable grid specialist.

Expand full comment
José Luis de la Fuente's avatar

Really outstanding review and analysis; thank you.

Your previous 2-part post on power system control and analysis was superb.

Expand full comment
Gene Nelson, Ph.D.'s avatar

Thank you for your analysis. I agree the large frequency oscillations are an indication of immanent grid failure. I believe the April 28, 2025 Iberian Peninsula blackout is an example of a "policy gird" failure per Meredith Angwin in her 2020 book Shorting the Grid: The Hidden Fragility of Our Electric Grid. https://www.amazon.com/Shorting-Grid-Hidden-Fragility-Electric-ebook/dp/B08KZ51SDP

Two perfectly serviceable nuclear power reactors were off line because the Socialist government of Spain was TAXING those plants to subsidize Spanish solar power. "How the Lights Went Out in Spain - The country flew too close to the sun—which is to say it relied too heavily on unreliable solar power." By Gabriel Calzada and Manuel Fernández Ordóñez, Ph.D., Updated April 30, 2025 4:01 pm ET https://www.wsj.com/opinion/how-the-lights-went-out-in-spain-solar-power-electric-grid-0096bbc7?st=dtf6np&reflink=desktopwebshare_permalink If the government wanted a stable grid, they would *not* be taxing the energy source that provides essential frequency stabilization services. CGNP discusses such policy questions in its popular March 4, 2024 article "Why is Grid Inertia Important? Without sufficient synchronous grid inertia, the grid becomes unstable and a blackout occurs," GreenNUKE Substack. https://greennuke.substack.com/p/why-is-grid-inertia-important

ElectricityMaps showed that after a few days, Spain had all but one of its seven nuclear power reactors back on line. The grid operator is curtailing wind at mid-day. (The remaining nuclear power reactor was in a refueling outage.) The oscillations appear to have gone away. Your article discusses several novel expensive approaches that possibly could enhance Spanish grid stability. I suggest the motto, "Keep it simple, stupid" (KISS) is a superior approach.

There is a far simpler solution. The Spanish government could admit their policy grid mistake. (Eventually, the mistake will become quite apparent.) However, the electorate is angry when they learn about policy grid mistakes. Just ask Gray Davis, who was recalled by California voters in 2003 in the wake of the ENRON - induced failure of California electricity deregulation. CGNP is working on an article regarding CAISO's failure to learn from the Iberian Peninsula blackout. The consequences could be far worse than the minimum of seven deaths associated with Spain's quest to run a grid with just solar and wind. Physics tells us such an anointed Utopian fantasy will not work.

Expand full comment
Kilovar 1959's avatar

Thank you Gene

Expand full comment
Tuco's Child's avatar

Great title: "How the Lights Went Out in Spain - The country flew too close to the sun..."

Expand full comment
Gene Nelson, Ph.D.'s avatar

Please read the April 30, 2025 Wall Street Journal article, "How the Lights Went Out in Spain - The country flew too close to the sun—which is to say it relied too heavily on unreliable solar power." The above link is a no-cost download link available to all.

Expand full comment
Wayne Findley's avatar

Also worth reading Kathryn Porter's take on the whole saga, at https://watt-logic.com/2025/05/09/the-iberian-blackout-shows-the-dangers-of-operating-power-grids-with-low-inertia/

The grid ecosystem has thousands of connected devices, of differing makes, models, and componentry. Each then has proprietary hence differing software algorithms to 'control' those disparate devices. Then add in ordinary grid operations such as switching linesÿ (changes lengths, impedance and damping characteristics), adding or removing reactive power corrections, and adding or curtailing generation. All of which is not communicated to IBR's, which have to detect those changes and then do - well who can exactly predict what?

It seems to me that in such environments the probability of Emergent Behaviour approaches 100%.

The textbook definition of emergence stresses that emergent behaviors cannot be predicted by component analysis. Which rather negates the regulatory effort to standardize them devices or at least their interfaces with Der Grid.

Your suggestions about much better measurement and transparency are spot on. Another possible policy is that IBR's need to communicate with each other, to lessen the evident tendencies to each individually detect, then try to correct, grid local grid anomalies, and thus jointly tend to over-react. This alone could explain some of the oscillations observed. Porter does mention this as well.

It is just fascinating, here on the other side of the world, seeing politics and physics collide in such a widespread way......

Expand full comment
Kilovar 1959's avatar

Great comment, thank you!

Expand full comment
Ed Reid's avatar

One has to wonder what the results of a Chinese "kill switch" test would look like.

Expand full comment
Kilovar 1959's avatar

We found the device, now to find the trigger. Once you find that you can do some testing.

Expand full comment
Sineira's avatar

Do you have a link to some info on these devices found?

Expand full comment
Ed Reid's avatar

https://www.msn.com/en-us/technology/renewable-energy/remote-controlled-blackout-hidden-kill-switches-in-solar-farms-spark-national-security-panic-in-u-s/ar-AA1ESMVK

Note US experience: "Recently, in the US, there was an incident where Chinese-made solar inverters were switched off remotely. Although there is no clear evidence of wrongdoing, it raised fears about the security of energy infrastructure."

Expand full comment
Tom's avatar

I wonder how much it would cost rate payers to implement your suggestions. I’ve heard some suggest that reliable solar is expensive even if the panels are free.

Expand full comment
Kilovar 1959's avatar

That’s been the major obstacle to renewables, all the extra you need to make them work. Gordan’s Knot at the link on the bottom talks about the cost of the synchronous condensers, The Energy Bad Boys have talked about the high cost of batteries, and the advanced programing IBR would be on the very cutting edge of new tech, so not cheap. This is why public funding is so often used, it pushes the cost out of the realm of what a rate payer can support.

Expand full comment
Sineira's avatar

Is it really major? Seems to me it's small to minor but you need to address it.

Batteries seem to be a good solution and the prices keep on dropping.

Expand full comment
Ed Reid's avatar

NREL estimates current 4-hour battery costs at $500 per kWh, which is projected to drop to approximately $250 per kWh by 2050. The Tesla Megapack stores 19,600 kWh at an installed cost of approximately $415 per kWh.

https://www.therightinsight.org/Current-Storage-Deficit

Expand full comment
Sineira's avatar

That seems like a very pessimistic forecast given the past trend and also not in line with other forecasts. But what do you expect from a partisan foundation? https://www.goldmansachs.com/insights/articles/electric-vehicle-battery-prices-are-expected-to-fall-almost-50-percent-by-2025

Expand full comment
Ed Reid's avatar

The DOE National Renewable Energy Laboratory is hardly a "partisan foundation". (That NREL projection is pre-Trump.)

Expand full comment
User's avatar
Comment deleted
Jun 1
Comment deleted
Expand full comment
Sineira's avatar

Well cost wise all the calculations I have seen point to it STILL being much cheaper than nuclear power or fossil fueled plants.

Plus with more individual plants you can get a more resilient network.

Expand full comment
Kilovar 1959's avatar

The pushback is you need to add so many layers, the generation itself, then the batteries with advanced inverters, then the rotating inertia, then the gas backup for multi-day loss of output, they all add up. Yes battery prices have fallen, but that's just one component. Also fallen is a relative statement like eggs are $9/dozen instead of $12/dozen. Lifespan is short for the high capital cost, 10 to 15 years on average for renewables and batteries. You put all those things together and that's a costly package.

Expand full comment
Gene Nelson, Ph.D.'s avatar

Excellent summary. Multiple expensive band-aids atop layers of band-aids. Without taxpayer-funded subsidies, the solar scheme collapses like a house of cards.

Expand full comment
Sineira's avatar

Not even close. The issue is that new nuclear and/or fossil fuel plants are very expensive and not trending down in cost.

But I'm sure you know that.

Expand full comment
Ed Reid's avatar

The "purists" would prefer "Green Hydrogen" to NG for longer outages.

https://www.therightinsight.org/The-Great-Green-Hope

https://www.therightinsight.org/Great-Green-Challenges

Expand full comment
Kilovar 1959's avatar

Hydrogen is a head scratcher for me. Everything i have seen is it takes more energy to make it that it provides when you use it, meaning it's a net energy sink. I know Robert Bryce is pretty outspoken about it.

Expand full comment
Sineira's avatar

There are plenty of countries with very high usage of renewables. The costs for those services just aren't that high.

I'm sure whatever is most economical will win and the long-term trends are very clear.

Expand full comment
Kilovar 1959's avatar

On that we can agree, it's long been my position that the government needs to get out and let the market decide. End the subsidies and all the back and forth rule swatting, and back door deals. BUT, it cannot be economics alone, it has to be reliable. Thank you for participating Seneira.

Expand full comment
Tom's avatar
Jun 12Edited

“There are plenty of countries with very high usage of renewables. The costs for those services just aren't that high.” I don’t believe this is so. Could you provide a list? Please include subsidies along with retail prices. Maybe someone has plot of total (rate plus subsidy) costs vs wind/solar penetration. Countries with lots of hydro would distort the results so should probably be omitted or tagged as such.

Expand full comment
depletedUranium's avatar

Well done.

This piece makes me want to open an old EE text book I picked up cheap years ago. But I have a job and a life.

The importance of inertia and how to minimize ocillations within an electric grid is obviously important.

Sidebar fact. There are ocillations and harmonics when machining precise holes. Coordinate measuring machines can show micron-level out-of-round conditions that "cycle" around a hole's circumference. That "cycle" often shows the machine's cutting tool diameter. Other harmonics may indicate part movement/instability along an axis during machining.

Expand full comment
Jeff Walther's avatar

Excellent analysis, although I'm rating it from a position of ignorance about power systems. I particularly liked this bit of artistry, "The Iberian system would behave much like a kite with no tale on the end of the weak transmission string. These periods of oscillation had been happening for several years."

I would like to blame it all on renewables, but your realistic analysis is certainly a better approach. Plus it reaches a similar conclusion in a round about way. If fixing renewables requires all that extra stuff, the cost is going to be astronomical.

It's like buying a bargain car and discovering that things like wheels and a throttle are after-market expenses.

Expand full comment
Tuco's Child's avatar

Very interesting and educational!

Expand full comment
World of Abundance's avatar

Isn't there an obvious way to make these situations less disastrous? You said Norway and Scandinavia are asynchronous due to an hvdc connection. Would it be good to divide some of these continent size grids into smaller asynchronous grids with such connections

Expand full comment
Kilovar 1959's avatar

Yes, there are things you can do, take a look at my post Power Systems – Where are My Controls Part 2. A well designed RAS scheme can react and split a system into pieces to isolate a big issue into a smaller area. I also suggested in the article the underfrequency load shedding should match the potential generation loss. Everytime something happens we learn, and we learn how to do things better, or at least we should.

https://open.substack.com/pub/kilovar1959/p/power-systems-where-are-my-controls-21f?r=23kggy&utm_campaign=post&utm_medium=web&showWelcomeOnShare=false

Expand full comment
Gary A. Abraham's avatar

Given the breadth of costly mitigations for a high-IBR grid, I’m looking for some analysis of emissions reduced by Spain’s system. That would round out our understanding of a policy failure on the grid.

Expand full comment
Richard Rusk's avatar

Thanks for following this topic. I know the ENTSO-E report has still not been published.

I think the Red grid was only 37% solar and wind before the blackout. I put screenshots from the April 28 graph.

What do you think? Is the quote of a much higher percentage not supported by this chart?

Here is my post on Substack

https://www.richardrusk.com/p/spains-grid-was-37-solar-and-wind

Here is my source

https://demanda.ree.es/visiona/peninsula/nacionalau/acumulada/2025-04-28

Expand full comment
Kilovar 1959's avatar

Your conjecture would certainly fly in the face of all the stories and comments I have read. What is your hypothesis for the collapse?

Expand full comment
Richard Rusk's avatar

Hi, Kilovar. My expertise is on the customer side of the meter. I don’t have a hypothesis.

I have an observation, though. I don’t think this was an event that should cause so much panic. And certainly this is not a time to say renewables obviously don’t work. The Spanish grid has been working now for a month with no reported issues that I know of. The blackout started at 12:30 pm on April 28 and by dawn on April 29 it was all taken care of.

I haven’t read about any damaged equipment so I have described this event as more like a circuit breaker popping open than a car crash. They had most of the blackout taken care of by 2 am, and every last bit by 4 am. A commenter somewhere I make comments said he was in Barcelona and on a mountain he got to by train, or something like that. I asked when the power came back on for him in Barcelona and he said 10 pm.

I know my way around the ERCOT dashboard and I know some Spanish so I went looking for a dashboard showing the sources on April 28. I found it. My reading of the graph and accompanying table is that the Red grid was not 70% solar and wind. That’s all. Just now I made a post on my Substack page showing the April 28 graph and the list of power sources. I have a link, too, so anyone else can go look. I could be wrong but I can’t get 70% out of the table or graph.

I have been reading articles and right now we seem to be in a finger-pointing mode among an association of renewable generators, an association of traditional generators, and the grid operator.

On May 15 PV Magazine ran an article about a cabinet minister reporting to the Congress of Deputies on what she knew. She said the investigation by ENTSO-E is not finished but the direction of the investigation is to find the source of the oscillations you mentioned.

When I read the ENTSO-E website there is a page saying that the results of the investigation won’t be released to the public until some broader group of grid experts in Europe have a chance to see it.

Expand full comment
Kilovar 1959's avatar

Well seven people died, that's pretty permanent, pretty serious. I think four from a direct loss of power to vents. Spain's economy is largely driven by the financial industry, which is dependent on reliable power. A twenty four interpretation in the flow of data would result in a multi-billion hit to the economy.

It’s really pointless to argue over the renewable content, all the official news outlets are very different than your conclusions. If you are right, then it will prove out in the end.

However making light of the impact of a nationwide blackout, on that I will stand my ground. Such events are simply unacceptable in modern society.

Regarding Spain now, by royal order Spain is now running more than 50% traditional generation and curtailing renewables. This will continue until the investigation concludes and remediations are in place.

Expand full comment
Richard Rusk's avatar

Thanks for the response. I am glad to make your acquaintance.

The April 28 blackout was a disaster and a tragedy for those killed, the people left behind, and many other disruptions to people’s lives, many of which will leave permanent damage to their health or finances. Companies may go bankrupt and the owners forced into bankruptcy, too. All the employees and their families will lose their income and perhaps permanent financial resiliency. I have lived through multi-day blackouts on the Gulf coast and they are damaging to society. The vulnerable people and weak businesses suffer most. I have lived it.

My conclusion about the renewable content is just from looking at the chart published by Red Electrica, with me looking at my laptop screen several time zones away. If I am misinterpreting it I am willing to be corrected. I put it on my Substack page with a link. I am hoping people will look at it and correct me if I am wrong. I see far less than 70% renewables as Inverter-Based Resources. It kind of looks to me like the 70% number came up somewhere and it took on a life of its own. I seem to be the only person who went to check on the source.

I have a comment about the panic over this outage as a blow to society. It will create a lot of non-grid decisions. One of my observations from the user side of the meter is that customers are slowly turning to generating their own power and making grid power secondary in their lives. The central role of the electrical grid in society is slowly fading. The April 28 blackout in Spain like the Feb 2021 freeze blackout here in Texas has changed a lot of customers’ feelings about depending on a grid. The grid people will strengthen the grid and some customers will make other arrangements. We will be better off as a society because of both actions.

We agree that a report with forensics and recommendations from ENTSO-E will be available eventually, hopefully soon.

Expand full comment
Gene Nelson, Ph.D.'s avatar

Per Bloomberg reporter Javier Blas, https://x.com/JavierBlas/status/1916857352197701963

Here are the readings five minutes before the blackout.

(IBR= inverter based resources. SGI = synchronous grid inertia, from turbine-derived generation.)

Energy Source MW

Solar PV 17,657

Wind 3,499

Total IBR 21,156

Solar Thermal 1,498

Cogeneration 1,356

Nuclear 3,387

Gas - Combined Cycle 982

Total SGI 5,725

Total 26,881

IBR to SGI Ratio 3.70

These statistics show a very 'brittle" grid prone to oscillate with inadequate SGI. There were numerous smaller-scale blackouts on the Iberian Peninsula at mid-day prior to the major April 28, 2025 blackout. The Spanish royal order to return to conventional generation and curtail solar and wind is preventing more blackouts.

Expand full comment
Kilovar 1959's avatar

The outages the day before kind of play into my thoughts that sloppy system operations were a major contributing factor. The Red Electra system seems to be run by Royal Decree, which really ties the hands of the professionals. Plus there have been a number of statements about “Spain has the most expensive power system in Europe”.,which speaks to arrogance and complacency. You can certainly have the fanciest machine in the business, but if you fail to operate it correctly, it will break.

Expand full comment
Richard Rusk's avatar

Thanks, Gene. I have figured out why I was misled in reading the fractions. Red Electrca quotes percentages that are not percentages of the total demand. I don't know the basis of the percentages they quote.

I read the X post by Javier Blas and made a comment on the post. He left off hydro at 3172 MW at 12:30 pm on April 28. I put in my comment a graphic from Red Electrica at 12:30 pm. The list is there.

I agree with your IBR at 21,156 MW. Adding in hydro to the SGI makes it 8897 MW. The ratio is 2.38. Do you agree hydro belongs in the total SGI?

Do you agree that the Red Electrica grid has been working every day starting with April 29?

Yesterday, June 6, at 12:35 pm was similar to the mix on April 28.

The total demand was 29061 MW. All the sources contributing over 1000 MW are:

solar 20522 MW, wind 1274 MW, combined cycle 3230 MW, nuclear 4140 MW, cogeneration and waste 1487 MW, hydro 1849 MW, and solar thermal 1438 MW.

So solar was 70.6% of demand and wind was 4.4% at 12:35 pm yesterday.

I am not an expert on grid operation. But I feel like I am the only one looking at the information directly from Red Electrica. Here ia link to the June 6 graph.

https://demanda.ree.es/visiona/peninsula/demandaau/acumulada/2025-06-06

Expand full comment
Sineira's avatar

Interesting details.

The lack of infrastructure to deal with oscillations is an issue.

We will see if that is the cause or not but even if it is, that is the issue and not the renewables themselves.

Expand full comment
Ed Reid's avatar

Conclusions from September-2022: surviving volatility? Thunder Said Energy

Similarly challenging is the second-by-second volatility of renewables. We spent time in September aggregating good data for solar (first chart below) and wind (second chart below). The purpose here is not to 'argue against' wind and solar. We simply need to understand these issues in order to find the best back-ups. That is the goal in our 20-page report.

https://thundersaidenergy.us20.list-manage.com/track/click?u=27881727130d8ad1056d0e618&id=e1b1b40bd5&e=4f525a183a

https://thundersaidenergy.us20.list-manage.com/track/click?u=27881727130d8ad1056d0e618&id=fba915a413&e=4f525a183a

Power grids are, in our view, the best way to backstop ALL forms of renewable volatility, from second-by-second to year-to-year. This complex and misunderstood industry is going to be worth $1trn pa in the energy transition (chart below). In September, we reviewed some of the most exciting long-distance transmission projects that are currently being progressed.

Battery implications? 75-80% of the short-term volatility events from wind and solar are <60-seconds in duration. Arguably, this makes them better-suited to being backed up with super-capacitors than lithium ion batteries?

Expand full comment