New supercomputer to accelerate research

UCAR > Communications > Staff Notes Monthly > November 2001

November 2001

New supercomputer to accelerate research

NCAR's new IBM SP supercomputer, which arrived at the Mesa Lab in early October, provides atmospheric scientists in Boulder and across the country with a staggering advance in computational, storage, and communications capabilities.

The first phase of the Advanced Research Computing System was delivered last month.

When fully operational, the system—known as the Advanced Research Computing System, or ARCS—is expected to accelerate research in global and regional climate change, droughts, short- and long-range weather prediction, wildland fires, turbulence, atmospheric chemistry, space weather, and other critical areas.

Electricians install the conduit for the electrical distribution to the new IBM SP nodes. (Photos by Carlye Calvin.)

"The addition of ARCS to NCAR's computing center is the single biggest increment in raw computing power in NCAR history," says NCAR director Tim Killeen. "It will provide U.S. scientists with speed, efficiency, and data storage space they need to stay at the forefront of climate, weather, and many other essential areas of research."

NSF, which provides the bulk of NCAR's funding, purchased the computer system to advance a wide range of research topics outlined in the agency's ten-year plan for the geosciences. The cost of the computer contract, including system upgrades, maintenance, and technical support over a three-year period, is $24 million.

NCAR is acquiring the system in three phases. This will allow SCD to maintain a stable, state-of-the-art production facility for the next three to five years—an important criterion for the successful development and refinement of atmospheric and earth-system models.

The first suite of equipment, delivered last month at the Mesa Lab, more than doubles the computational capacity of NCAR's current IBM SP to a peak of two trillion floating-point operations per second (two teraflops) and increases current disk storage capacity fivefold, up to 10.5 trillion bytes of data. This part of the system is undergoing tests and should be available to users this month.

A second delivery, in September 2002, will introduce the next generation of IBM processors, nodes, and intercommunication switch technology, adding nearly five teraflops at peak operation. The package will also include 21 terabytes of new disk storage. In the fall of 2003, NCAR will receive IBM's next round of switch technology, whose lower latency and higher bandwidth will significantly increase the performance of parallel research models.

Specifications aside, what this boils down to is that NCAR's computer power is expanding exponentially. The first phase of the ARCS system doubles the capacity of the current system, and it represents a sixfold increase in capacity over the original IBM RS/6000 SP system that arrived at the Mesa Lab just more than two years ago.

Sophisticated models

The leap in supercomputing power is critical for scientists striving to understand the complex interplay of physical, chemical, and biological processes that drive earth's weather. As scientists construct increasingly sophisticated models of the climatic effects of the atmosphere, oceans, land surface, and sea ice, they require ever-growing computer capabilities.

Consider, for example, the role of newer and more powerful supercomputers in climate change research:

The Community Climate System Model, a computer model of earth's climate, simulates a century of climate in 29 days.
NCAR's current IBM SP system is capable of running 6 such simulations simultaneously in that time.
The ARCS equipment installed last month increases the number to 14 such simulations in a month.
A year from now, when the second phase of new equipment kicks in, the combined power of NCAR's IBM SP systems will allow nearly 50 one-century climate simulations per month.

Numerous NCAR staffers were involved in the procurement. They include Peter Fox (HAO), Tony Craig and Jim Rosinski (CGD), Stacy Walters (ACD), Bill Skamarock and John Michalakes (MMM), Betty "Charlie" Benmark, Beth Stansberry, and Pat Munson in the Contracts Office, and an SCD team of Aaron Andersen, Bill Anderson, Tom Engel, George Fuentes, Steve Hammond, Gene Harano, Robert Harkness, Al Kellie, Jeff Kuehn, David Mitchell, Bernie O'Lear, Pete Peterson, and Dick Valent. Assisting in the effort was an evaluation team that included external experts. The team was comprised of Jim Hack (CGD), Bert Semtner (Naval Postgraduate School and chair, SCD Advisory Panel), and James Kinter (executive director of the Center for Ocean-Land- Atmosphere Studies of the Institute of Global Environment and Society and chair of the Climate Simulation Laboratory Allocations Panel).

"This was actually the first time that the NCAR divisions had representatives on the supercomputer procurement team and the first time the procurement was reviewed by an external evaluation team," says Beth, who is director of contracts and sponsored agreements for the Contracts Office. "It was an extremely large effort by a lot of people."

NCAR's agreement with IBM includes early access to new hardware and software technologies, as well as new software features and tools development, training for systems engineers and operators, and specialized training tailored to the needs of the user communities served by NCAR.

The next generation? In the wings is IBM's Blue Light High- Performance Computing project, a research effort to develop future supercomputer systems with speeds exceeding one petaflop, which is one thousand trillion floating-point operations per second. NCAR's collaboration with IBM on Blue Light will provide valuable input on hardware and software design and the applicability of new computer architecture to research in the atmospheric and related sciences. Such a system will not become a reality for several years, but it may eventually significantly advance, and potentially revolutionize, computer modeling of climate, weather, and other earth systems.

And then, computer experts agree, scientists will press for even more powerful machines. For example, climate modelers want the capability to produce higher resolution climate models, as well as fully coupled models that incorporate the chemistry and biology of the atmosphere, continents, and oceans.

"That's been the history of supercomputers: the work will expand to exceed the capacity of the machines," says SCD's Tom Engel. "The scientific horizons keep getting pushed further out with every new technology that's introduced. I don't think we'll ever get to the point where scientists say, Gee, there's nothing left to study."

• Anatta and David Hosansky

In this issue... • Other issues of Staff Notes Monthly
UCAR • NCAR • UOP

UCAR > Communications > Staff Notes Monthly > November 2001

Edited by David Hosansky, hosansky@ucar.edu
Prepared for the Web by Jacque Marshall
Last revised: Thu Dec 20 16:57:22 MST 2001