For SmartPlanet this week I offered a model to demystify oil prices. The most important factor is one you’ve probably never heard of, while the ones you hear about aren’t that important. I also showed why unconventional oil from fracked shales will reinforce, not relieve, the pain of high oil prices.  Read it here: A model of oil prices

Everyone wants to know why gasoline prices are high, and going higher still, while the U.S. remains in the throes of recession. Yet the usual explanations—threats by and against Iran, inventory levels, speculators, and so on—strangely avoid the most fundamental and obvious reason: supply and demand.

Perhaps this is because getting current data on supply and demand, then making sense of it, can be a challenge. Blaming speculators requires no data literacy. Inventory numbers are easy to get and analyze. And it takes no effort at all to build a story around a headline.

In a decade of studying oil data, I’ve seen these kinds of panics and heard the same misguided interpretations over and over. So I know that if you really want to understand why oil prices are what they are, you need a much more sophisticated model which accounts for supply and demand, along with a great deal of additional complexity. Today I’ll share a conceptual model I created which, while by no means comprehensive, should give you a much better understanding of oil prices both in the past and the future.

But before I do that, I will explain some important concepts for the uninitiated.

Price

The first thing to understand is that oil is a global market. The gasoline prices you see in the U.S. are affected by both domestic and world supply and demand. U.S. data only gives you half the picture.

In the chart above, I have used the benchmark for oil prices known as Brent Crude, a trading classification for various grades of “light sweet” crude that come from the North Sea. Of the more than 150 crude benchmarks, Brent is a reasonably good marker for global oil prices. The U.S. benchmark, known as West Texas Intermediate or WTI, is not a good global price marker because it reflects transportation congestion at its trading point in Cushing, Oklahoma. This congestion has resulted in a local storage glut which depresses its price.

Supply

Oil supply can be characterized in many different ways, and supply data can vary substantially from source to source depending on how it is defined. I recently explained why “peak oil” should be understood as a phenomenon of crude oil only, but in this article I will use the more generally-accepted “all liquids” definition of supply which includes non-oil components such as synthetic liquids made from tar sands, and natural gas liquids associated with crude oil. The supply data in this model comes from the U.S. Energy Information Administration (EIA), mainly because they conveniently offer current, global data in a monthly format, and exclude biofuels.

The most recent supply data is for November 2011, when world oil production was 88.18 million barrels per day (mbpd).

Spare capacity

No matter how strong oil demand is, there is always some amount of “spare capacity,” or the ability to produce more oil at will. Spare capacity can result from oil being too expensive to catch a bid, or from being too “heavy” or “sour” for “simple” refineries to process it, or from producers simply turning off a few taps in order to support prices, or from other exogenous events.

Only Saudi Arabia still has a substantial amount of spare capacity. Exactly how much they have is a state secret, but according to the Paris-based International Energy Agency (IEA), the kingdom had 2.03 mbpd of official spare capacity in January 2012, or about 1 mbpd of realistic spare capacity on a sustained basis. This conflicts with repeated assurances by the state-owned oil company Saudi Aramco that the kingdom currently has about 12 mbpd of total production capacity and is only producing 9 mbpd, giving an implied 3 mbpd of spare capacity, but I don’t take that number at face value and I don’t know of any serious analysts who do.

IEA identifies a smattering of additional spare capacity from other OPEC producers—0.28 mbpd from Kuwait, 0.17 mbpd from Angola, plus smaller amounts elsewhere—to bring the OPEC “effective” spare capacity up to 2.82 mbpd. This total excludes potential spare capacity from Iraq, Nigeria, Venezuela, and Libya due to civil war and other geopolitical considerations that would prohibit new production from coming to market any time soon.

For this model, I am using the OPEC “effective” spare capacity data, which was laboriously compiled from monthly IEA reports by Dow Jones reporter James Herron, to whom I owe a debt of gratitude for sharing it with me. (None of the usual oil data reporting agencies offer a compiled, years-long monthly data series for spare capacity.) But that’s really a notional number, which I have used mainly because I can get the data. The reality could be much worse. Bear in mind that the 3.58 percent of spare capacity you see for November 2011 in the above chart, equivalent to 3.16 mbpd, actually could be a mere 1 mbpd if Saudi Arabia turns out to be the only real swing producer. Indeed, at the OPEC meeting in December, Saudi Oil Minister Ali Al-Naimi told Bloomberg that “there was no excess supply in world crude markets and that his nation had been adjusting output to match fluctuating demand of recent months.”

Outside the OPEC cartel, there isn’t any real spare capacity. Non-OPEC producers, including Russia and the U.S., have been producing flat-out for many years. According to EIA data, non-OPEC production peaked in 2009 at 55.8 mpbd and then declined to 52.7 mbpd in 2011, but still contributes about 60 percent to world oil supply.

In this model I use spare capacity, expressed as a percentage of total world oil supply, as an indicator of how stressed the oil market is.

The model

As you can see in the chart, supply and demand are the fundamental drivers of oil prices. When the balance between supply and demand is tight, the “call on OPEC” rises and spare capacity falls, shown here as the blue line. Generally speaking, spare capacity is inversely related to price.

However, spare capacity doesn’t always have an immediate effect on price. From 2002 through 2004, spare capacity fell sharply from 8 percent of supply to about 2 percent, but oil prices barely budged. It took awhile for the recognition that the market had tightened to sink in. It wasn’t until peak oil arrived at the end of 2004—when the production of global crude oil (only) flattened out below a ceiling of 74 mbpd—and China’s demand simultaneously began its breathtaking ascent, that spare capacity fell into the danger zone below 1 percent. This kicked off a new era of permanently higher prices, and by 2006 the price of crude had tripled to $60 a barrel.

Thus we can postulate that there is a spare capacity threshold at which oil prices suddenly become sensitized. I have drawn it here as a dotted blue line at 2.5 percent of supply, but other levels could be argued as well. (For reference, two and a half percent of global oil supply in November was 2.2 mbpd, just a little more than the current official Saudi spare capacity.) When spare capacity falls below that threshold, prices have tended to rise, and when it rises above that threshold, prices have tended to fall.

I submit that the spare capacity threshold is the most important factor in understanding oil prices. But it is not the only one.

Headline risk

A few of the major events that oil traders and pundits fingered as key drivers of price are shown in dark red callouts on the chart. No doubt they did exert some pressure, both upward (hurricanes and OPEC production cuts) and downward (the financial collapse of 2008). But over time, headline risk is temporary. Once the shocks wear off, the market moves on and starts looking at fundamentals again. Headline risk also tends to be more localized.

For example, consider the effects of Hurricane Katrina, which hit Louisiana on August 29, 2005, and Rita, which hit on September 23. WTI fell by $6.65 from August to November while Brent fell $8.74, perhaps reflecting that the larger concern in the U.S. was about adequate supply after the Gulf Coast systems were knocked offline, whereas in Europe the concern was more about lower demand. As we can see, their effect was temporary as prices resumed their upward trajectory in December. That was dictated by the fundamentals of supply: spare capacity was still below the 2.5 percent threshold.

When spare capacity started rising again, nearly touching the threshold at 2.4 percent in June 2006, the oil rally began to lose steam. By August it was over, and prices fell from $73.23 back to $53.68 in January 2007. Then spare capacity began to fall again, and prices began moving back up. Long before the reality of the financial crisis had entered the public consciousness in 2008, commodity traders had realized that Something Big was happening. The tells were already there in the spare capacity data.

Monetary policy risk

The effect of monetary policy on oil prices is significant, if hard to measure. Eliding that complicated subject for now, I’d just note that two rounds of “quantitative easing”—in which the Federal Reserve printed trillions of U.S. dollars out of thin air to stimulate the economy—have undoubtedly pushed up oil prices. Oil is not only a globally traded commodity, but one that is almost exclusively traded in U.S. dollars. The price of a barrel of oil has to go up when trillions of dollars of new money are suddenly dropped on the market, simply to preserve its value.

This is certainly at least a partial explanation for why oil prices have continued to move up since late 2008 even as spare capacity broke above the threshold. Both QE1 and QE2, as shown on the chart, were followed by sharp increases in oil prices in an era of stable and significant spare capacity.

The narrow ledge

As I explained last week, another important factor is the “narrow ledge” of prices.

Producers generally need at least $60 to $70 a barrel to bring a new barrel of cheap oil capacity (in places like Saudi Arabia) online today. A barrel of new capacity from marginal resources like tar sands, the Bakken and Eagle Ford shales, the Arctic, and ultra-deepwater wells in the Gulf of Mexico or off the coast of Brazil might set you back $80 to $90 a barrel. And that’s just the price producers need to turn a profit. Social welfare obligations of the sort offered by Venezula, and social stability bribes offered in Saudi Arabia and Russia since the dawn of the Arab Spring, increase those nations’ needs for increased oil revenues. That pressure will ultimately, if indirectly, find its way into the price of Brent.

At the same time, there is a limit to how much pain consumers can take. We know that $100 oil (WTI) and $4 gasoline is the kiss of demand death in the U.S. That level corresponds with about $115 for Brent, mainly because they run a far more efficient fleet of vehicles in Europe which travel far shorter distances.

The twin thresholds of the narrow ledge, shown here between $70 and $100 a barrel and starting in 2009, may partially explain why prices have fallen along with spare capacity since April 2011. It may also explain why spare capacity stayed constant just over 6 percent through 2009 and 2010; there was neither sufficient incentive to bring marginal production capacity online, nor was the price low enough to cause supply destruction.

Where oil prices are headed

At this point you are probably wondering why I haven’t mentioned speculation as an important price driver. That’s because I don’t believe it is one. Many hours of studying technical forensic studies on the influence of speculators during the 2008 price spike have revealed their real influence: When spare capacity drops below or rises far above the threshold, it creates tradeable opportunities and speculators rush in. They are probably responsible for only the last 10 to 15 percent of the price in an upward or a downward spike. Everything else can and should be explained according to the principles of the model.

On Monday this week, the London-based Centre for Global Energy Studies revealed that according to its research, oil speculators have been in “hibernation” and that noncommercial speculators (those who trade oil speculatively, rather than on behalf of a refinery) have been moving out of the market since late October. Fundamentals have been driving the market, as Asia continually outbids the West for a declining amount of available exports. U.S. inventory numbers are but a removed indicator of that.

Spare capacity is firmly in the driver’s seat now, and savvy traders are watching it closely. Analysts from Barclays Capital, quoted in the same piece, said that sustainable spare capacity has fallen below 2 mbpd and warned against being on the short side, flagging headline risk. And in October, JP Morgan said: “Our analysis suggests that supply constraints will again be reached by the end of 2013, driving a quarterly rise in our ICE Brent price forecast to $130/bbl.” Morgan’s call was hardly bold; it was backed by over a year of steadily declining spare capacity. It was also a modest price target, as $130 is not far above the $123 peak of April 2011, when official spare capacity was at 4.67 percent—again, the most recent official number we have is 3.58 percent for November.

On the current trajectory, shown in green in the model, spare capacity will break under the 2.5 percent threshold by mid-2012. At that point, if prices fall below $115 due to the narrow ledge restriction, a new price spike will be likely.

So forget about blaming high gasoline and oil prices on speculators, and remember that headline risks are temporary. What we have here is a fundamental supply problem that isn’t going away; in fact, it’s going to get progressively worse as the plateau of oil production ends and turns down into inevitable decline. Prices will repeatedly bump between the floor and ceiling of the narrow ledge, as driven by spare capacity. Unconventional oil from fracked shales will reinforce, not relieve, the pain of high oil prices. The only way out of this junkie’s hell is to get off the needle.

Chart by Chris Nelder