I have vocally and repeatedly proclaimed that computer models cannot prove anything. As I was working on updating my 5-day Engineering course I came across a perfect example of what I'm talking about.
This model started with the best dataset ever assembled. Really. The best ever. The underlying data was all "coincident" to this model. By that I mean that the people collecting the data had a strong motive to make it as accurate and complete as it could possibly be. Those folks got paid and they paid partners and mineral owners based on the data (so they had no significant incentive to illegally adulterate it), oh yeah, the data is required by law to be complete and accurate and has been since the 1940's. Further the data was collected each month on upwards of 400,000 discrete entities operated by nearly 100,000 business entities, all with an explicit license to operate that is not trivial to acquire. In other words the data entering this model has had financial and legal incentives to be accurate and complete. Of course, the dataset is monthly U.S. gas production by well.
If you start with this high quality data and bring in:
[ul]
[li]Historical wellhead price and consumer price data sets along with an independent forecast of those prices into the future[/li]
[li]A detailed data set containing historical new-well permits that can be compared to the price data over time[/li]
[li]A detailed data set containing new facility permits that can be compared to new well permits and the price data[/li]
[li]A detailed list of issued permits for facilities (that take up to 10 years to build after the permit is issued) with their projected completion dates and projected capacities (see the big uptick in the attachment in the Alaska data in 2019 representing the pipeline coming on line)[/li]
[li]Independent forecasts of inflation[/li]
[li]Historical and (independently) projected steel pipe worldwide manufacturing tonnage and prices[/li]
[li]A team of very talented, very experienced Engineers, Economists, Statisticians, and Computer Modelers[/li]
[li]A project deadline that the team felt was very liberal[/li]
[li]No limits on budget for manpower, computing equipment, or software[/li]
[/ul]
It really doesn't get any better than this. They published the attached forecast in the 2007 Energy Outlook there were some glitches in the first version so they updated it and published the attached in the 2008 Energy Outlook. This chart was reprinted hundreds of times over the next few years. I haven't seen it much since 2011. I pulled in the data yesterdat and added the actual production between 2006 and 2011 (last data available that breaks out the various gas types).
So at year 5 you find:
[ul]
[li]Unconventional gas under predicted 92% (using the Unconventional Gas forecast as the denominator)[/li]
[li]Onshore conventional over predicted 41%[/li]
[li]Offshore over predicted 60%[/li]
[li]Alaska over predicted 55%[/li]
[li]Total gas under predicted 19%[/li]
[li]If you remove the Unconventional component, total would be over predicted 55%[/li]
[/ul]
This is at year 5 of a 24 year forecast. In early 2008, Gas prices were over $10/MSCF, the drilling in the Marcellus, Hanesville, and Fayetteville shales had accelerated and rig counts were approaching all-time highs. All of this data was readily available to the modelers, but they didn't quite believe it and tweaked the model back to a slight increase followed by flatish with offshore taking up the slack.
I don't mean to ridicule these guys, they did a workmanlike job. I made a similar blunder in 1990 when I failed to include a group of wells (that I had already built pipe to) in a forecast of the value of a company that was on the market. A competitor did include those wells and offered $15 million more than we did--the group of wells I excluded produced that much profit the first 6 months and 23 years later they are still on production.
My point is that with a superb set of clean data, unlimited time and budget, a team with all of the requisite skills and no incentives for reaching a particular conclusion couldn't predict something as "simple" as gas production within 55%, how can anyone put any credence in the climate models that have questionable data, intense time pressure, intense budget pressure, and intense pressure to reach a specific conclusion? Hell, they could even be "right", but I won't be willing to accept that until we can look back at a body of predictions that have the same shape as the actual (raw) data for that period. So far we are not even close.
David Simpson, PE
MuleShoe Engineering
"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"
This model started with the best dataset ever assembled. Really. The best ever. The underlying data was all "coincident" to this model. By that I mean that the people collecting the data had a strong motive to make it as accurate and complete as it could possibly be. Those folks got paid and they paid partners and mineral owners based on the data (so they had no significant incentive to illegally adulterate it), oh yeah, the data is required by law to be complete and accurate and has been since the 1940's. Further the data was collected each month on upwards of 400,000 discrete entities operated by nearly 100,000 business entities, all with an explicit license to operate that is not trivial to acquire. In other words the data entering this model has had financial and legal incentives to be accurate and complete. Of course, the dataset is monthly U.S. gas production by well.
If you start with this high quality data and bring in:
[ul]
[li]Historical wellhead price and consumer price data sets along with an independent forecast of those prices into the future[/li]
[li]A detailed data set containing historical new-well permits that can be compared to the price data over time[/li]
[li]A detailed data set containing new facility permits that can be compared to new well permits and the price data[/li]
[li]A detailed list of issued permits for facilities (that take up to 10 years to build after the permit is issued) with their projected completion dates and projected capacities (see the big uptick in the attachment in the Alaska data in 2019 representing the pipeline coming on line)[/li]
[li]Independent forecasts of inflation[/li]
[li]Historical and (independently) projected steel pipe worldwide manufacturing tonnage and prices[/li]
[li]A team of very talented, very experienced Engineers, Economists, Statisticians, and Computer Modelers[/li]
[li]A project deadline that the team felt was very liberal[/li]
[li]No limits on budget for manpower, computing equipment, or software[/li]
[/ul]
It really doesn't get any better than this. They published the attached forecast in the 2007 Energy Outlook there were some glitches in the first version so they updated it and published the attached in the 2008 Energy Outlook. This chart was reprinted hundreds of times over the next few years. I haven't seen it much since 2011. I pulled in the data yesterdat and added the actual production between 2006 and 2011 (last data available that breaks out the various gas types).
So at year 5 you find:
[ul]
[li]Unconventional gas under predicted 92% (using the Unconventional Gas forecast as the denominator)[/li]
[li]Onshore conventional over predicted 41%[/li]
[li]Offshore over predicted 60%[/li]
[li]Alaska over predicted 55%[/li]
[li]Total gas under predicted 19%[/li]
[li]If you remove the Unconventional component, total would be over predicted 55%[/li]
[/ul]
This is at year 5 of a 24 year forecast. In early 2008, Gas prices were over $10/MSCF, the drilling in the Marcellus, Hanesville, and Fayetteville shales had accelerated and rig counts were approaching all-time highs. All of this data was readily available to the modelers, but they didn't quite believe it and tweaked the model back to a slight increase followed by flatish with offshore taking up the slack.
I don't mean to ridicule these guys, they did a workmanlike job. I made a similar blunder in 1990 when I failed to include a group of wells (that I had already built pipe to) in a forecast of the value of a company that was on the market. A competitor did include those wells and offered $15 million more than we did--the group of wells I excluded produced that much profit the first 6 months and 23 years later they are still on production.
My point is that with a superb set of clean data, unlimited time and budget, a team with all of the requisite skills and no incentives for reaching a particular conclusion couldn't predict something as "simple" as gas production within 55%, how can anyone put any credence in the climate models that have questionable data, intense time pressure, intense budget pressure, and intense pressure to reach a specific conclusion? Hell, they could even be "right", but I won't be willing to accept that until we can look back at a body of predictions that have the same shape as the actual (raw) data for that period. So far we are not even close.
David Simpson, PE
MuleShoe Engineering
"Belief" is the acceptance of an hypotheses in the absence of data.
"Prejudice" is having an opinion not supported by the preponderance of the data.
"Knowledge" is only found through the accumulation and analysis of data.
The plural of anecdote is not "data"
