The utility industry is being disrupted on every side, prompting worries about its stability.

A new policy paper from the Edison Electric Institute (EEI), an association of shareholder-owned U.S. electric companies, details the “disruptive challenges” the sector faces. These private, for-profit companies, also known as investor-owned utilities or IOUs, serve about 70 percent of the U.S. population. They are usually subject to different regulations than publicly owned utilities or co-ops, and must turn enough profit to retain their investors in a difficult, constrained, low-margin business.

Here are just a few of the challenges the industry is facing.

Falling costs of distributed generation

The cost of power generation from solar PV, wind, geothermal, micro-hydro, and fuel cells running on natural gas has been dropping dramatically. Residential and commercial utility customers can now generate some or all of their own power economically instead of drawing it from the grid. The cost of such distributed generation is set to continue falling as more of it is deployed around the world, and “could directly threaten the centralized utility model,” the report acknowledges.

Distributed generation costs are falling too rapidly for lumbering utilities to adapt to these new business realities. In Germany, which has deployed renewable power aggressively for the past decade, solar power has nearly reached grid parity and may now be “unstoppable” even without subsidies, according to Macquarie Group, a global investment bank. Germany’s coal-fired and nuclear power generators are now struggling to remain profitable as their share of the market shrinks as higher-priced peak hours of the day are increasingly met by solar.

Worse, being able to generate your own power means that you might eventually decide you don’t need the grid at all. The EEI raises concerns that “the longer-term threat of fully exiting from the grid (or customers solely using the electric grid for backup purposes) raises the potential for irreparable damages to revenues and growth prospects,” observing that it may become difficult to recover investment costs over a 30-year period, as it has done in the past.

Increasing use of demand-side management technologies

The cheapest watt is the one you don’t have to generate, and the most expensive watt (for fossil-fueled power) is the one that has to be generated at peak times. As such, improving efficiency and shifting loads away from peak times has made good sense for consumers and the economy as a whole. But those aren’t necessarily good things from a utility’s standpoint.

Efficiency gains have gradually reduced the amount of power customers use, and “spending on energy-efficiency programs will increase by as much as 300 percent from 2010 to 2025, within a projected range of $6 billion to $16 billion per year,” according to research by Bloomberg New Energy Finance. This investment will have “a meaningful impact on utility load and, thus, will create significant additional lost revenue exposure,” the EEI report says.

And now there’s a new threat: demand-response technologies and “curtailment service providers” (CSPs), which offer voluntary programs that let customers reduce their power consumption at peak times in return for lower, off-peak electricity rates. Some demand response customers will cut their power consumption when the utility requests it — for emergency response, or to reduce peak load — while others will voluntarily reduce their demand when advised by CSPs, in order to reduce their overall power costs.

Demand response is one of the reasons why power consumption is decreasing overall, which means that utilities aren’t able to make as much money on peak power generation or increase their revenue by building new power generation capacity.

Government programs to incentivize selected technologies

The utility industry complains about “selected technologies,” but only when those technologies — like renewables, efficiency upgrades, and demand response — cut into their fossil-fueled power generation business. They don’t complain about the government picking winners when it sells coal mining leases on public lands at well below fair market value, or when it continues the 100-year-old percentage depletion allowance which lets oil and natural gas companies write off a certain percentage of the oil and gas they extract.

The declining price of natural gas

The boom in U.S. shale gas helped to drive the price of natural gas down from $13/MMBtu in 2008 to $2/MMBtu in 2012, which reduced the price of wholesale power and hurt utility profits. As the ratings agency Fitch noted in February, low gas prices put pressure on “power producers that depend heavily on the sale of excess power to subsidize their retail revenue,” and over the long term, it can accelerate the retirement of coal-fired power plants whose capital costs were recovered long ago. Ultimately, it may force utilities to spend significant capital on newer, cleaner natural-gas fired plants in an environment of relatively low wholesale power revenues.

Capital expenditures required to upgrade the grid

The U.S. power grid is old, creaky, largely analog, and suffering badly from decades of deferred maintenance. This is precisely what one would expect when it is operated by for-profit companies, which have every incentive to “maximize shareholder value” by squeezing every last month of service they can out of their capital assets. They are required by law to maintain certain service levels, but they have not been required to proactively upgrade their equipment in order to, for example, adapt to a more robust, security-oriented digital age. Indeed, as a recent New York Times article noted, “some utilities don’t know if a customer has lost power unless that person calls to complain,” adding that “[m]any utilities still rely on paper maps of their systems that become outdated quickly.”

Market forces, the spread of technologies like distributed generation and smart meters, and growing regulatory pressure are now forcing the utility sector to face its deferred maintenance backlog, and those costs are going up. This has the utilities worried, because it is coming at a time of industry contraction.

Slowing economic growth trends

Thanks to the recession, improved efficiency, and distributed generation, electricity demand in recent years has been “anemic.” The power industry is no longer growing as it did in the past, but now it must ” deploy capital investment at almost twice the rate of depreciation to enhance the grid and address various regulatory mandates.”

IOUs are now facing a “vicious cycle,” EEI says, wherein the industry’s decline will make it harder to pass on the costs of providing service, because customer rates are tied to usage. As usage declines, the costs of new investment must be passed on to a shrinking pool of customer demand, which in turn forces per-unit prices higher still. As those prices rise, investment in efficiency and renewables becomes even more cost-effective, which shrinks usage further. Ultimately, these dynamics could leave a small number of customers supporting the costs of a large chunk of grid infrastructure, leaving utilities with “stranded investments.”

It could become very difficult for IOUs to remain profitable, which will test the loyalty of investors. Brokerage firms are forecasting earnings per share of 4 percent to 7 percent for IOUs, EEI notes, but if the utilities aren’t able to meet these investor expectations, “a wholesale reevaluation of the sector is likely to occur.”

While all of these challenges to the traditional private-sector utility model are indeed disruptive, it’s instructive that they’re also for the good of the environment, and for our communities. Nobody ever said that energy transition was going to be easy, or that there wouldn’t be losers as well as winners. Some utilities will navigate the transformation successfully, while others will fight it tooth and nail until they die. It may make sense for some of them to convert to public entities. Stay tuned to this space for more on that.

***

Chris Nelder is an energy analyst and consultant who has written about energy and investing for more than a decade. He is the author of two books (Profit from the Peak and Investing in Renewable Energy) and hundreds of articles, and has been published by Scientific American, Slate, the Harvard Business Review blog, Financial Times Alphaville, Quartz, the Economist Intelligence Unit, and many other publications.