All wet

Well, it is that time of year again – NOAA has just released their annual prediction for the upcoming hurricane season. And boy, will it be a doozy! They are predicting 12 – 14 named storms [1], 8 – 14 hurricanes[2], and 3 – 7 major hurricanes [3].

But, before we panic [4], it is worthwhile to ask ourselves “Do these ivory-tower yabbos actually know what they are talking about?” Fortunately, we have the tools available to answer that question: statistics!

Over the past eleven years, the folks at NOAA have predicted the number of named storms, the number of hurricanes, and the number of major hurricanes. That gives a total of thirty-three predictions. For each prediction, they can be too high, too low, or just right. That’s three possibilities. If they are just guessing (or doing no better than chance), then they should get 11 (=33/3) of their predictions right. If they are doing better than that, then we can assume that they actually know what they are doing.

So let’s compare the predictions (stored on the NOAA site) with the actual values (as given by Wikipedia). Charting them up, we get this:

In 17 (52%) of the predictions, they were right. That is definitely better than chance, so we may safely say that our tax dollars are well-spent when they go to NOAA.

What is interesting about the results is that they are too high (over predict) at about the same rate that they are too low (under predict). As you may recall from our previous foray into statistics, that is symptomatic of a gaussian normal function, which implies that the errors are due to random noise and not to bias. Thus, their models are good, and will improve as they find out how to squeeze more of the noise out of the system.

How can we tell that the models are improving? Look at the range of values on the named storms. Notice how it is about ten storms this year? That is the one sigma prediction; they are about 70% sure that there will be that many storms. If the range goes down, then that implies that the models are getting better (more precise).

Now, if you will excuse me, I have to go set up a second hurricane kit. Somehow, I don’t think that just one will do me this year…

John

[1] Think “Whopping great thunderstorm”. Millions in damages, very few displaced, almost no casualties.
[2] Think “Ike” (or Ike). Tens of millions in damages, a few hundred displaced, very few casualties.
[3] Think “Katrina” or “Andrew“. Billions in damages, thousands displaced, hundreds dead.
[4] Don’t Panic!

About these ads

16 thoughts on “All wet

  1. Granted, this sampling is far too small for any really deep analysis, but there seems to be a pattern to the over and under predictions. Lately they've been over predicting. They also seem to be in a trend of slightly increasing the sizes of the ranges which increases the odds of being right. You'll also notice that in 2009 they hit the minimum of the range for named storms and just the year before they hit the top of the range. The current prediction for named storms is actually 14-23. In fact, the ranges in this report are nothing short of huge.And let's also not forget that they NEVER venture to predict where, when, or even if any of those storms will make landfall or how strong they will be if they do.I'd take these numbers with a grain of salt, but it's always best to be prepared so make sure you are!

  2. the ranges in this report are nothing short of huge.

    Which implies huge uncertainty. Maybe I should have brought that out more?
    John

  3. Which implies huge uncertainty. Maybe I should have brought that out more?

    Uh, yeah. And let's not forget that dirty little secret of forecasting I learned in grad school. Meteorologists (NWS, TV, etc.) don't put out those long range forecasts because they have confidence in the models (though they are getting better) they do it because the public demands it. Anything beyond 36-48 hours (maybe 72 these days) is really pretty much a crap shoot.

  4. I've thought about looking at these numbers a slightly different way. First off, we should ignore 1999 and 2000 because they used a different forecasting system. By keeping their numbers low they increase their odds of looking smart.For the remaining years let's try reducing the ranges to the median value. In the case of named storms they only got it right in 2007 with a now equal number of years too high and too low. For the number of hurricanes they do slightly better, but they are still almost as likely to over forecast as to under forecast or get it right. (I'll let them keep those years of being right as it is really kinda tricky to have only half a hurricane.) For the number of major storms only 2008 gets shifted to a miss which leaves them with a record of 3-3-3. Are you still sure you want to call this a success?

  5. Meteorologists (NWS, TV, etc.) don't put out those long range forecasts because they have confidence in the models (though they are getting better) they do it because the public demands it. Anything beyond 36-48 hours (maybe 72 these days) is really pretty much a crap shoot.

    Here you are wrong. They may not be able to tell the track of the storm more than three days out, but they can determine which factors for hurricane formation are favorable and which are unfavorable. And from that, they can predict the overall storm level.For example, nobody in their right mind thought that last year would be a good storm year, because of el nino's strength. This year, we're into a la nina, which implies more storms. And then there are sea surface temperature, dust production in Africa, and other factors – all of which can be known well in advance and all of which influence the number of storms.

    John

  6. First off, we should ignore 1999 and 2000 because they used a different forecasting system.
    I wondered about that – the reporting was different.For the remaining years let's try reducing the ranges to the median value.

    No. Providing a range is a valid way to predict events, and has a long and honorable history in science. Heck, that's how I got my MS!
    John

  7. No. Providing a range is a valid way to predict events, and has a long and honorable history in science. Heck, that's how I got my MS!

    But the range keeps changing. If it were getting smaller with time I'd view that as increased confidence in the models, but it's getting larger. That issue needs to be addressed.

  8. Meteorologists (NWS, TV, etc.) don't put out those long range forecasts because they have confidence in the models (though they are getting better) they do it because the public demands it. Anything beyond 36-48 hours (maybe 72 these days) is really pretty much a crap shoot.

    Here you are wrong.

    No, I am not. This is not your area of expertise in spite of the fact you like to think it is. I learned this in grad school first hand from the people who are actually making the forecasts. I'd like to see you argue this point with them.

  9. But the range keeps changing.
    No, it doesn't. Take another look at the data. There are four years with an uncertainty of 4 storms (2006, 2005, 2004, 2001), four years with an uncertainty of 5 storms (2008, 2007, 2003, 2002), and one year with an uncertainty of 6 storms (2009).

    Similarly, the uncertainty in the number of hurricanes does not appear to be changing. There are four years with an uncertainty of 4 hurricanes (2009, 2008, 2007, 2003), and five years with an uncertainty of 3 hurricanes (2006, 2005, 2004, 2002, 2001).

    Finally, the uncertainty in the number of major hurricanes also appears to be stable. There is one year with an uncertainty of 4 major hurricanes (2008), six years with an uncertainty of 3 major hurricanes (2009, 2007, 2006, 2005, 2004, 2003), and two years with an uncertainty of 2 major hurricanes (2002, 2001).

    No, I am not. This is not your area of expertise in spite of the fact you like to think it is. I learned this in grad school first hand from the people who are actually making the forecasts. I'd like to see you argue this point with them

    Yes, you are. Your degree was nearly twenty years ago, and you have obviously not kept up with the improvements in the field. And I'm not the one arguing with you; Dr. Gerry Bell, Dr. Jae Schemm, (of the NOAA Climate Prediction Center), Eric Blake, Todd Kimberlain, Dr. Chris Landsea, Dr. Richard Pasch, (of the National Hurricane Center), and Stanley Goldenberg, (of the NOAA Hurricane Research Division) are. They claim that we can make these predictions, and do so.

    Here are a few references discussing how these predictions are made and evaluated – go do your homework and then come back to the discussion when you are prepared for it:

    Bell, G. D., and M. Chelliah, 2006: Leading tropical modes associated with interannual and multi-decadal fluctuations in North Atlantic hurricane activity. J. of Climate. 19, 590-612.
    Chelliah, M., Bell, G. D., 2004: Tropical multi-decadal and interannual climate variations in the NCEP/ NCAR Reanalysis. J. Climate, 17, 1777-1803.
    Elsner, J. B., Schmertmann, C. P., 1993: Improving Extended-Range Seasonal Predictions of Intense Atlantic Hurricane Activity. Wea. and Forecasting, 9, 345-351
    Goldenberg, S. B., C. W. Landsea, A. M. Mestas-Nuñez, and W. M. Gray, 2001: The recent increase in Atlantic hurricane activity: Causes and implications. Science, 293, 474-479. Gray, W. M., 1984: Atlantic seasonal hurricane frequency: Part I: El Niño and 30-mb quasi-bienniel oscillation influences. Mon. Wea. Rev., 112, 1649-1668.
    Gray, W. M., 1984: Atlantic seasonal hurricane frequency: Part II: Forecasting Its Variability. Mon. Wea. Rev., 112, 1669-1683. Knaff, J. A., 1997: Predicting summertime Caribbean pressure in early April. Wea. and Forecasting, 13, 740-752.
    Landsea, C. W., Bell, G. D., Gray, W. M., Goldenberg, S. B., 1995: The Extremely Active 1995 Atlantic Hurricane Season: Environmental Conditions and Verification of Seasonal Forecasts. Mon. Wea. Rev., 126, 1174-1193.
    John

  10. Here are a few references discussing how these predictions are made and evaluated

    I'm not as out of date as you think I am and most of the papers you referenced predate the time I moved on. Did you read this that I linked to in my first comment? They openly discuss the high levels of uncertainty in their predictions. Maybe you and I see this differently because your arguments are based on what you've read in official documents while mine is based on face to face discussions with the scientists themselves. The exact same words can take on different connotations in those settings. Body language and tone of voice carry a lot of information.

  11. Did you read this that I linked to in my first comment?

    Duh. Where do you think I got my information from in the first place? Go back and re-read it yourself. They discuss the uncertainty – but do not say that it cannot be done. They actually list the factors that can be tracked this far out and that act as proxies for the number of hurricanes. As their track record shows, they are on the right track.

    Maybe you and I see this differently because your arguments are based on what you've read in official documents while mine is based on face to face discussions with the scientists themselves.

    And so are mine. I've met some of the folks who do this at NOAA, and had frank and open discussions with them about the quality of their predictions. The real difference between you and me on this is that I'm including the peer-reviewed literature, and you seem to be ignoring it.

    John

  12. I vote for no weather events this year. There is already oil on the beach at Gulf Shores, Alabama this week. Small blobs in the water stick to your skin if you stand in the surf. The ring at the top of the tide line with sea weed is full of bigger sandy globs. No many birds in site but I do not know if this is usual or not. We will not be here long enough to eat seafood but I expect the prices to go through the roof now. Will have to stick to catfish and walleye.

  13. I vote for no weather events this year. There is already oil on the beach at Gulf Shores, Alabama this week. Small blobs in the water stick to your skin if you stand in the surf. The ring at the top of the tide line with sea weed is full of bigger sandy globs.

    As an oilman, please accept my apologies. I do not work for BP, and am severely upset with their lack of foresight. The spill was preventable. Now it is screwing up the economy, the ecology, and my future.If it is any consolation, a good hurricane might be just what the doctor ordered for increasing the amount of bioremediation. It would spread the oil out and break it into smaller bits, increasing the surface area and thus speeding up the process of microbes eating the oil.
    John

  14. I wondered what the reason for dispersants was. Being a pessimist I thought it would just hide the problem. I am amazed government agencies are standing in the way of many of the cleanup possibilities because they would not remove a high enough percentage of the oil. I would think if you could remove any oil it would be a good thing. One of my great-uncles had an oil well in his backyard in the 50s. It was very messy. He said when it quit producing they cleaned up. I am sure the oil companies know how.

  15. I wondered what the reason for dispersants was.
    It depends on whom you ask. BP would probably say that the purpose is to keep the oil from forming large slicks [1]. An radical ecologist would probaly say the purpose was to keep the full extent of BP's culpability form coming ot light [2]. And the government would probably say that the purpose was to distract from capping the oil [3].
    John
    [1] Which it does. The dispersant is basically a type of soap, which gets rid of grease (such as oil) by forming little containment capsules around the grease molecules and holding them in suspension in the water.
    [2] Which it does. By keeping the oil in suspension, it makes it appear as if there has been less oil spilled.
    [3] Which it does. By starting a fight over whether or not to use the dispersant (much less in the quantities that are being used), BP has managed to get its opponents to fight among themselves. This then allows BP more breathing room to fix the root problem of the spewing oil well.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s