February 2008

A closer look at today’s forecast

NCAR researchers study how people use weather predictions

It’s been more than 40 years since Americans were first introduced to probabilities of precipitation. Instead of simply conveying a threat of rain or snow without using numbers, forecasters began to include percentages, as in “a 60% chance of showers.”

Decades later, most U.S. residents still aren’t precisely sure what these probability-of-precipitation forecasts mean. However, they do get the general idea—and many of them want more such uncertainty information.

These are among the insights arising from a major survey conducted by NCAR’s Societal Impacts Program. SIP involves scientists from SERE, RAL, and ESSL, with funding from the U.S. Weather Research Program.

“Weather forecasts are unavoidably uncertain,” notes Rebecca Morss, “but most public forecasts come with, at best, limited information about their uncertainty.”

Rebecca and SIP colleagues Julie Demuth and Jeff Lazo are sifting through the extensive data gleaned from their 2006 survey. They made several presentations at last month’s American Meteorological Society meeting, and two papers are in the works, with more to come.

One of the group’s goals is to provide grounding and perspective for the National Weather Service (NWS) and other providers who are now contemplating how best to improve public weather forecasts. The idea of the survey arose after a 2006 National Research Council report on estimating and communicating uncertainty in weather and climate prediction. Rebecca served with Barb Brown (RAL) on the panel that produced Completing the Forecast, which was sponsored by the NWS. (See “On the Web.”)

“Meteorologists often find it challenging to communicate uncertainty effectively,” Rebecca says. She and her colleagues found that relatively little research on communicating the uncertainty in ­weather forecasts had been published since the 1980s. “Being on the NRC panel helped clarify for us that this was an important priority area that hadn’t been adequately addressed,” she says.

Late in 2006, SIP conducted a nationwide, controlled-access Internet-based survey, with more than 1,500 U.S. respondents. The sample was checked to ensure geographic, demographic, and ethnic diversity; respondents came from every U.S. state and had similar gender and race characteristics to the public at large. The survey questions drew on previous research, asking people how important weather is to them as well as how they regard, interpret, and use forecasts.

Would you like probabilities with that?

Thanks to the progress of weather research and the advent of new technologies, there’s now a basis for creating far more detailed forecasts, including shades of gray that don’t fit into the usual template. For example, say that forecasters predict a 20% chance that a cold front will arrive soon enough to hold tomorrow’s high down to 70°F, but otherwise they expect temperatures to soar to 85°F. How best to present these options in a single outlook? This scenario made for one of the survey’s more complex questions. More than 90% of the respondents preferred something other than a flat forecast of 85°F, and many liked knowing why the forecast was uncertain. Among seven alternatives offered in the survey, this was the favorite option: “The high temperature tomorrow will most likely be 85°F, but it may be 70°F, because a cold front may move through during the day.”

People tend to read between the lines of deterministic forecasts, those in which the uncertainty isn’t made clear. When asked what a forecast high of 75°F actually means, the largest share of respondents in the SIP survey (more than 40%) picked the range 73–77°F. Only about one in twenty people took the 75°F forecast literally. “These and other results show that most people have some understanding of relative uncertainty in forecasts,” Julie says.

Several questions were designed to see how people might take to different ways of presenting forecasts. When asked whether they’d prefer a TV weathercaster who presents single-value predictions (such as a high temperature of 75°F) versus one who gives ranges (74–78°F), about twice as many people (45%) chose the latter. About a quarter of the respondents were comfortable with either choice.

“The majority of people liked information about uncertainty in a forecast, and many preferred it, at least in the ­scenarios we tested,” Julie says.

Several other lines of research are emerging from the survey. For example, Jeff is leading an analysis of how much people use and value the forecasts they now get. He says that the average household gets forecasts 115 times a month. After accounting for the few survey respondents who don’t use forecasts, and extrapolating to the public at large, this means that Americans access more than 300 billion forecasts a year.

Follow-up work found that the median annual value placed on forecasts was $285 per household, or about $32 billion nationwide. Another line of research based on the survey responses, with psychologist colleague Alan Stewart at the University of Georgia, examines people’s weather salience—essentially, how important weather is to them and their lives—and how that relates to their use of forecasts.

As for the age-old question of whether it’ll rain on tomorrow’s picnic, the study found that people interpret a percentage verdict—“a 60% chance of rain”—in many different ways. “Some of these interpretations are more sophisticated than meteorologists might expect,” Rebecca says.

As in most previous studies, only a small fraction of people chose the explanation closest to the one typically used by meteorologists: “It will rain on 60% of the days like tomorrow.” The most popular choice: “60% of weather forecasters believe that it will rain tomorrow.” Responses to related survey questions suggest that, as Julie puts it, “most people don’t know the technical definition of probability of precipitation, but many seem to have built sufficient understanding of it through experience.”

Analysis of the survey data is ongoing, and a variety of interdisciplinary research questions in this area remain to be addressed, according to the SIP team. It will take several years for new types of forecasts to be developed, possibly including innovations such as new icons to express uncertainty. However, the results to date give SIP scientists the sense that the public is ready for the next level of weather predictions. Just as a motorist needn’t understand every detail of a car’s engine in order to drive safely, weather consumers should be able to draw added value from new types of forecasts even if they never look under the meteorological hood.•

On the Web

NCAR Societal Impacts Program

Completing the Forecast (2006 NRC report)

In this issue...

A closer look at today’s forecast

Internship programs gear up for summer

The heart of winter

NCAR/UCAR media office readies staff for interviews

Short Takes

Just One Look

Staff Notes home page | News Center

©UCAR | Privacy Policy | Terms of Use | Contact Us | Visit Us | Sponsored by