Ask Weather Dan: The Old Farmer’s Almanac

I have a confession to make. I’ve never read The Old Farmer’s Almanac. You might make the assumption that it’s wise to keep up with the competition, and while I do that on a regular basis, I’ve never honestly paid much attention to The Almanac.

December 14, 2011; 6:00 AM • by Weather Dan

I have a confession to make. I’ve never read The Old Farmer’s Almanac. You might make the assumption that it’s wise to keep up with the competition, and while I do that on a regular basis, I’ve never honestly paid much attention to The Almanac. I’ve always assumed that their “forecasts” were something akin to the daily horoscope–so generic that they can easily be construed to be correct in any number of ways, yet nearly impossible to show anything statistically.

Published every year since 1792, The Old Farmer’s Almanac is the longest continuously-published periodical in American history.

One of the driving forces behind the continued popularity of the publication has been its long-range weather forecasts. On the menu of their website, the weather section is third on the list, just below “Home” and “Almanac Store,” and current weather forecasts get center stage on their frontpage. They have a section offering “instant access to our best content” featuring the 2012 long-range forecast, and a “customized weather history,” available to subscribers willing to pay $4.95 for 30 days of access. The press release announcing the 2012 edition touts their “80-percent accurate weather forecasts.”

The Alamanac‘s own online history tells the story of Roger Scaife, who became Almanac editor in 1936, and is said to have committed the “Greatest Almanac Blunder of All Time.” Scaife decided to replace the forecasts in the Almanac with simple precipitation and temperature averages. During his three-year tenure, circulation dropped so sharply that he was fired and the publication was put up for sale.

In 1942, the FBI arrested a German spy on Long Island, fresh off a U-boat, with a copy of the Almanac in his pocket. The publishers, concerned that their weather forecasts would be considered aid to the enemy, reached an agreement with the US government to call the forecasts “weather indications,” avoiding any potential legal trouble with Washington.

The Almanac lists Michael Steinberg as their meteorologist, mentioning that he holds degrees in both atmospheric science and meteorology. However, a cursory search through both LinkedIn and Google fails to find any more information about Mr. Steinberg or his credentials. The OFA also lists AccuWeather, Inc. as a provider of “weather graphics and consultation.” Given that a Michael Steinberg is also listed as a Senior Vice President at AccuWeather, I suspect it is safe to say this is likely the same person. [Clarification: Information from multiple sources confirms that Mr. Steinberg is the man responsible for the forecasts in the Almanac. In addition to being SVP at AccuWeather, he holds a degree in atmospheric science from Cornell University, a masters in meteorology from Penn State, and has been a member of the American Meteorological Society for more than 40 years.]

The publishers of the Almanac discuss their weather prediction methodology. This, too, is somewhat generic, stating that their current forecasting methods come from their founder’s notes, which are locked in a black box at their headquarters in New Hampshire, on forecasting using solar activity. These are then augmented with modern science, advances in solar science, climatology, and meteorology. Their forecasts are, they claim, based on variations from the climate averages published by the National Oceanic and Atmospheric Administration. This same information is repeated within the almanac, along with a mention that the most recent climate data cover the 30-year span from 1971 to 2000. New climate normals covering the period from 1981 to 2010 were released back on July 1st. In other published interviews, editor Janice Stillman has disclosed that their forecast is based on finding similar periods of solar and sunspot activity in the past and assessing the meteorological conditions at that time.

Their forecasting methodology separates the continental United States into 16 regions. Portions of Virginia are in four of them: Richmond and most of Virginia east of the fall line lie in region 2; Hampton Roads, Emporia, and other points along the eastern VA/NC border lie in region 4; areas of the state from the Piedmont, across the Blue Ridge and Alleghany Mountains, lie in region 3; and the Virginia highlands in coal country on the Cumberland Plateau fall into region 7. Some of these regions are quite large: 13, for example, stretches from North Dakota to Texas, while 14 reaches from central Washington into northern New Mexico. Alaska and Hawaii, despite earning mentions on the website, get no love in the print version.

Listed on page 200 of the 2012 Almanac is their analysis of their forecast for the winter of 2010-2011, which they advertise as 90.6% accurate. How they determine this with such precision is not disclosed, but their basic methodology is published and evaluates the direction (positive or negative) of the average and actual average temperature relative to the climate normals. They also feature a table showing one city from each of the 16 regions, along with their forecast departure from the average as well as the actual departure. They were correct in the direction of their forecast in 15/16 instances, and differed in magnitude by no more than 0.6 degrees. Given the size of some of these regions, it’s possible that the data they list may have been cherry-picked to accentuate successes and give the illusion that there were no failures.

Based on their own analysis, where they say that in 13 of the 16 regions, “we were correct in the direction of precipitation departure for normal in all or much of the region, or 81 percent accurate.” Thirteen divided by sixteen is indeed 0.8125, or 81 percent. However, they’ve reached this point by setting the bar almost as humanly low as possible. They count being correct in terms of direction in “much of the region” a forecast success, but don’t specify exactly what qualifies as “much of the region.” Could it be only 50%? Could it be 90%? They don’t say.

Region 2 stretches from just south of Richmond along the coast and the I-95 corridor, all the way to Boston and Cape Cod. Their forecast comes with a seasonal summary and then a breakdown of average temperature and liquid precipitation for each month from November 2011 to October 2012. Forecasts for this region include ones such as “Rainy periods, warm” (Nov 1-5), “Blizzard” (Jan 30-31), and “sunny, cool” (June 23-30).

Like I said, we’re setting the bar pretty low here. It’s hard to say exactly what’s required for this forecast to verify. I imagine it’s pretty hard to find a station in the entirety of region 2 that didn’t record precipitation at any point during those five days. Richmond recorded a trace of precipitation on the first and the fourth, and temperatures all five days were anywhere from one to eight degrees below the average for the day. The coldest of these days was 55 degrees, which I suppose isn’t exactly cold, so maybe it qualifies as “warm?”

The bigger point here is that we’re stuck arguing semantics and not numbers–or anything more concrete. When you set the bar this low, it’s hard to say you got a forecast wrong. So, I’m not sure there’s enough salt in Saltville, Virginia to be taken with these forecasts, especially the detailed ones. There may be a little more skill in the long-term, seasonal forecasts, but with the information that’s hand picked for us by the Almanac, even that’s hard to say with certainty.

Jan Null, a Certified Consulting Meteorologist and former National Weather Service forecaster, has published several independent evaluations of The Old Farmer’s Almanac weather forecasts. His most recent review of the 2009 OFA winter forecast, selected 5 of the regions created by the OFA and evaluated their temperature and precipitation forecasts from November 2008 to March 2009, assigning a letter grade based on performance. His results: 11 As, 6Bs, 6Cs, 3Ds, and 23 Fs.

Ultimately, The Old Farmer’s Almanac is great for a lot of things. It’s got great information on things like wintertime gardening, recipes, and astronomical and planting tables, but I wouldn’t ever use it as a weather reference.

December 14, 2011 6:00 AM 5