the valleys of
death and lost opportunities:
Toward an Earth Information System
When society invests in applied research, it has a right to expect
some benefit, whether in the form of an improved economy, a safer way
of life, or some other social gain. Over the past couple of years I
chaired a National Research Council (NRC) committee whose goal was to
look into improved ways of transferring NASA satellite research results
and technologies to NOAAs weather and climate prediction efforts,
in order to accelerate the rate of return to society of the investment
in research. Our report has just been published (see On
the Web ).
The difficulties in bringing new science and technologies into operational
use in industry, government, and academia are well known. Some people
have used the metaphor crossing the valley of death to describe meeting
these challenges, as in an earlier NRC study on weather satellites and
numerical weather prediction chaired by Eric Barron (Pennsylvania State
University) and James Mahoney (U.S. Climate Change Research Program).
(See On the Web.)
Our committee defined the valley of death as the graveyard for technologies
with known applications that fail to materialize. In other cases, the
research and user communities may not even recognize the potential uses
of technologies. We defined this graveyard as the valley of lost opportunities.
Together, these two metaphors describe the chief hazards in the complete
research-to-operations transition process, also dubbed the transition
Bridges are necessary to cross the valleys, and each bridge is composed
of building blocks. These include a solid research foundation, laboratories,
equipment, computers, algorithms, information technologies, and related
infrastructure necessary to support a robust pathway.
Depiction of the transition pathways from NASA research to NOAA operations.
In order to emphasize the importance of satellite research and operational
use of satellite data, and to strengthen the transition pathways between
research and operations, our report described the importance of weather
and climate to society, how weather forecasts and warnings are improving,
and how these improvements are leading to new users and applications.
Reinforcing the conclusions of a number of earlier NRC studies, as well
as strategic planning documents from NASA, NOAA, NSF, and others, our
committee looked optimistically to a future in which information about
weather and climate in particular, and about the Earth system in general,
will become increasingly valuable for an ever-expanding number of uses.
The road to an Earth
Information System (EIS)
Where might a stronger transition pathway lead us? We envision observations
and forecasts of the atmosphere, oceans, freshwater, land surfaces,
ice, biosphere, and space environments forming the basis for an Earth
Information System. This EIS will be a four-dimensional gridded set
of quantitative, geo-referenced data that describes the entire Earth
system in one temporal and three spatial dimensions.
Much like the Internet and the World Wide Web, we see the EIS as an
evolving, widely available resource. Like the early days of the Internet
and the Web, an informal prototype version of EIS exists today in the
data and analysis archives now available online (for example, at NOAAs
National Climate Data Center and NCAR). But this is not yet a total
systemit remains incomplete, unconnected, and difficult to use
compared to our vision of the future EIS. As EIS partners update the
system and make it more easily accessible, it will become increasingly
valuable and will support a larger and broader range of users. These
will include the research community, government at all levels, industry,
national security activities, and the public as a wholethereby
supporting the economy and welfare of all countries.
Much has been written on the impact of weather and climate on society.
According to some estimates, up to 40% of the approximately $10 trillion
U.S. economy is affected by weather and climate annually. One of the
major success stories of science during the 20th century has been the
extraordinary progress in understanding the complex atmosphere-Earth
system and our ability to observe and predict it on temporal scales
ranging from minutes to seasons. But there is still much room for improvement.
Our committee believes such improvements will come from the same successful
formula that has led us to todays weather and climate servicesa
balanced investment in research, better and more observations, and faster
computers. However, the rate of progress and return on the public investment
could be much faster if greater attention were paid to the transitioning
of research into operations.
Perhaps the area most in need of improvement is our use of the powerful
but complex observations gathered by radars and satellites. By themselves,
these dataespecially the billions of raw observations produced
by radars and satellites per dayprovide little direct value to
many users. For example, the raw data from radars are frequencies and
amplitudes of reflected radio waves. The raw data from infrared or microwave
satellite sensors are radiances emitted from the atmosphere below. The
raw data from GPS radio occultation measurements are the Doppler-shifted
frequencies from GPS signals. The full benefit of these observations
is wasted unless they can be first processed into more physically relevant
data (e.g., radar reflectivities, motion of precipitation particles,
temperature, and humidity), and then into information that people can
usewhether directly or through specific products created from
the data and information base.
A powerful way of processing the billions of observations and turning
them into information is assimilation of the observations into advanced
models of the Earth system (today mainly atmosphere and ocean models,
but in the future models of the entire Earth system). For example, radiances
from infrared satellite sensors and bending angles from GPS receivers
can be assimilated in global numerical weather prediction models to
produce more accurate forecasts and analyses of temperature, water vapor,
pressure, winds, and precipitation on a global grid. These analyses,
in turn, can be used for a variety of purposes beyond weather forecastsany
application that requires weather or climate data.
Yet compared to the resources put into the observing systems and weather
and climate services, the resources put into data assimilation and other
ways of using the observations are quite small. For example, U.S. agencies
invested more than
$4 billion in meteorological operations and related research during
fiscal year 2002. It is difficult to estimate how much is being invested
in data assimilation research, but my rough guess is about $20 million,
or only about 0.4% of the total. This investment is spread broadly and
thinly in universities, research laboratories, and operational centers.
It supports a large number of efforts related to the troposphere, stratosphere,
ionosphere, oceans, chemical tracers, and the carbon cycle.
We need a greater and more focused effort in order to better assimilate
data from current and planned observing platforms.
A faster transition
The present global observing system is already quite useful, but it
has developed in an ad hoc way. It could be far more useful with modest
investments in strengthening the transition pathways. The EIS will be
a promising and important outgrowth of this process, one that will support
society and foster enterprise in myriad ways.
How do we reach this goal? Instead of merely gathering more observations,
we need observations with the right variables and with the
right characteristicsthose in situ and remotely sensed
observations that together help create an accurate and complete EIS.
For example, two independent observing systems by themselves might each
produce temperature observations with a 2°C (3.6°F) error on
the average, but when used together and assimilated in a model, the
resulting temperature analysis might have an error of only 1°C (1.8°F)
or even less. At the same time, we must consider how limits on resources
will dictate the optimal global observing system. Finally, we must learn
how to most effectively use the resulting observations to produce the
EIS, upon which the greatest potential benefits to society will
depend. To accomplish this, the research and operational communities
and the end users will need to communicate amongst each other as never
before. Rick Anthes