UCAR > Communications > Staff Notes > February 1999 Search


February 1999

Special Report: UCAR and Y2K
Y2K resources and answers

You can't turn to your favorite news medium these days without encountering a story about the year 2000 problem (also known as the millennium bug). The Y2K problem stems from decades of programming practice that conserved memory and keystrokes by representing years with two digits instead of four (e.g., 99 rather than 1999). This worked fine from the dawn of modern computing until now. The problem with switching from "99" to "00" at the end of this year is that the second number is smaller than the first. Computers that aren't programmed to recognize 00 as 2000 may assume that anything that happens in 00 took place before 1999 (i.e., in 1900).


Someone said that the year 2000 problem is about 85% hype and 15% real. And the essence of the problem is that we don't know which 15% is real.

--Steve Dickson (F&A)

The result may not matter to your word-processing program. But what if a two-digit date embedded in a microchip or software code interferes with NCAR's supercomputers or the electronics on our aircraft sometime in the new year? Will our telephones work? What about the elevators or the electronic building-entry system? And what about our payroll system--how will it handle the rollover to 2000?

These questions have been on the minds of F&A's Steve Dickson and the seven other members of the UCAR Y2K Task Group at least since the end of 1997. With Steve as project manager, the group has developed a working document designed to keep UCAR moving forward on Y2K issues (see UCAR Web sites). The plan is part of UCAR's response to notification by NSF to "take all steps necessary to mitigate potential [Y2K] problems." Those steps must be taken within existing funding. How do you equip a multifaceted organization like UCAR to mitigate potential Y2K problems with no additional funds? Staff around the organization are making room by putting other projects on hold. "We just kind of squeeze it in," Steve says.

So far, that strategy is working. "Most of the problems," Steve notes, "the ones costing gazillions of dollars to fix that everybody's talking about, are COBOL code in large-production operations like banks, manufacturing, or utilities. . . . We don't have any of that in this organization.


I am not concerned about our ability to continue delivering data and computational resources to the UCAR community--outside of things that are beyond our control.

--Tom Engel (SCD)

"Our goal is to have a successful transition of all mission-critical activities," says Steve. The consensus of the task group is that what's truly critical to UCAR's mission is the performance of the scientific supercomputers and related user services. Health and safety and the payroll system are also high on the list. If problems arise with systems that aren't on the mission-critical list, they can be handled with contingency planning. For example, Steve explains, "If we have to, we can go in the back room with a typewriter" to produce accounts payable checks. In some cases, if Y2K compliance cannot be assured, software or hardware may be retired to avoid problems when the date rolls over.

Who's in charge?

The task group's charge from the UCAR President's Council was to facilitate the planning phase and produce quarterly status reports. The group is not in charge of operations. Responsibility for the technical work is distributed throughout UCAR. You can expect Y2K assistance in your own area to come primarily from the systems administrator(s) in your division or program. Task group member Tom Engel, who manages the High Performance Systems Section in SCD, thinks distributing the responsibility makes sense. "The system we have is more timely, more accurate, than a centralized system would be--especially in crisis situations." Steve agrees: "We could have made this a monster technical project, and instead we made it a management planning project. I feel like that's absolutely the right thing to do in this organization."

What's happening in SCD?

New SCD director Al Kellie was the project director for Y2K compliance at the Atmospheric Environment Service (the Canadian weather service), which was identified as a mission-critical system for the Canadian government. His budget was roughly Can$25 million and when he left, "the count was about 360 FTEs involved." However, Al adds, "I am not charged with the same responsibilities at NCAR, nor am I proposing the same degree of zeal at SCD. . . . This is a research institute. If we have to go down for a short period of time it's not as calamitous. We haven't been identified as part of the [government's] mission-critical structure. But it's very important that we do as thorough a job as we can so we can continue to provide services."


It's a chaotic system, just like the climate, where small perturbances in one part of the world can cause calamitous effects in other parts of the world.

--Pete Peterson (SCD)

SCD's plans for the supercomputers and other systems in its purview can be found on the Web. The first step was to identify all equipment with potential problems. The next step was to obtain manufacturers' upgrades.

"The target is April to have all our systems upgraded to vendor-supplied, Y2K-compliant software," says Tom. While all the supercomputers have been upgraded, a few other systems might not make that target. "The time we have to become compliant keeps decreasing, and yet it's only been in the last one to three months that our major vendors have released Y2K-compliant systems and product sets. So we're trying to play this balancing game between becoming compliant and not destabilizing our production environment."

Waiting for vendors to supply upgrades has slowed the installation and testing process--not just for the supercomputers, but for desktop systems as well. For the many people running Microsoft's Windows NT, for example, Tom notes that Version 4.0, Service Pack 3 "was proclaimed Y2K compliant when it came out," but then some noncompliant features were identified. "I anticipate that the same thing is going to happen with Service Pack 4. And there may be patch sets that we'll have to obtain from Microsoft [to fix that]."

Testing, testing

Even after systems are upgraded, they still need testing to see how they'll perform with the clock set to the year 2000. SCD has proposed a testbed to accomplish that task, not only for its supercomputers and other systems, but for other UCAR users as well. SCD's Network Engineering and Technology Section (NETS) is proposing a dual-purpose testbed using "hot spares" already attached to existing networking systems. (Hot spares are run attached to production systems so they can be tested continuously and upgraded with new software whenever the production system is upgraded.) The proposed configuration would allow anyone in UCAR to connect to the testbed.

Basil Irwin is Y2K project leader for NETS. He says the current proposal "is scaled down but offers SCD testbed facilities that will be available for others." The new design uses equipment in the production network while keeping test traffic logically isolated from production traffic. Some money may need to be spent, according to Basil, to test host-based services like DNS (domain name service, a part of Web administration) and e-mail. (See UCAR Web sites for more on the testbed proposals.)


Nobody that's honest will tell you that the power grid definitely will not fail. Hopefully the probability is close to zero, but no one knows how close to zero it is.

--Basil Irwin (SCD)

The NETS testbed is not designed to put the supercomputers through their paces. That's the responsibility of the Supercomputing Systems Group. According to SSG's year 2000 Web pages, the group will acquire a small SGI Origin 2000 for system testing. The plan is to configure that computer as a stand-alone testing environment. To test how CRAYs handle Y2K will require removing one from production and configuring it for testing only.

Basil is disappointed in the lack of additional funds for testing. "My personal opinion? It makes me wonder how serious [NSF] is about really fixing things. The institutions that are serious about fixing this problem have spent hundreds of millions of dollars." Here at UCAR he sees a lot of effort to address potential problems at the grassroots level, but a lack of oversight. "I'm seeing bottom-up, but not top-down. There are lots of isolated pools of people dealing with it in a noncoordinated fashion." On coordination, Steve Dickson concurs: "It's not clear in all cases who's got operational responsibilities, and maybe that needs to be more centrally coordinated."

Think globally, act locally

Grassroots efforts are indeed under way around UCAR. Here's a sample of what's being done to prepare for this coming New Year's Eve.

ATD: UCAR Y2K Task Group member Susan Stringer is a software engineer in ATD. "We're trying to inventory everything we have," she reports. Some of their computers, scientific equipment, and software were purchased off the shelf and some were built by ATD staff. "We have embedded processors in some of our equipment that someone put there 15 years ago and [that person] is no longer here." Once they've figured out where the problems are, they'll prioritize their mission-critical systems and start the upgrading process.

Although staff are spread thin, Susan isn't expecting a crisis. "We'll keep plugging away with the resources we have and see how it works. A lot of our problems are probably in our postprocessing software, where the date 00 will show up, but it's not going to stop the system."

Ron Ruth manages data for RAF. Most of the instruments on the NSF/NCAR aircraft are analog sensors that transmit voltages in real time. "We don't have a lot of date stuff going on." Ron is confident that in the few cases where dates are involved, fixes will be uncomplicated. RAF custom-built the data acquisition system for the aircraft it operates. "We have the ability to convert to four-digit dates when we do ground processing of resulting data sets," Ron explains.

RAF is examining two systems on the aircraft themselves: the inertial navigation system and the Global Positioning System (see It's not just the year 2000). Both are crucial for flying long routes over the oceans. "We've called the manufacturers and they've said 'We're not worried about it, but we're still working on it,' " says chief pilot Henry Boynton. "That's their business, so I'm sure they'll have fixes for it. It doesn't make any sense to worry about it." Still, Steve Dickson doesn't want to take chances with the aircraft, even if they're upgraded in time. "We probably will avoid flying at midnight on December 31st because the consequences of us being wrong are not tolerable. The risk is too great," Steve says.

ACD: Division administrator Brad Crysel, who also serves on the Y2K Task Group, has been making sure their administrative systems are compliant. "Our division director, Guy Brasseur, has made it an area of emphasis." They've purchased machines from the same vendor within the past year, at least partially to ensure Y2K compliance.

ACD systems administrator Tim Fredrick emphasizes the distributed nature of the problem. "Fixing the Y2K problem is really something that happens at many different levels. We're not in a position to fix models or applications that scientists have developed, but we are in a position to apply vendor updates to the systems on which those models run." Tim is hopeful that staff in the division will take a look at what they're responsible for and communicate with him. He points out that "each programmer who is responsible for code in ACD and at NCAR will need to take a careful look at that code to fix Y2K problems."

Tim has completed a good deal of testing on the division's central systems. He's hoping to attach clones of ACD's two primary servers to the NETS testbed "to further prove to myself that those systems will be able to provide the services on January 3 that they provide today."

UOP: Unlike NCAR's research divisions, most of UOP's activities were spun up in the past decade, when Y2K awareness was already starting to emerge. Thus, they have a head start on compliance. "We've talked to every program in UOP," says Kathy Strand, manager of budget and administration for UOP and the contact person for the office's Y2K effort. "Some problems were identified, but they're being taken care of. Overall, we are in good shape and feel as if we've checked things out."


We will try to touch base on everything we have. Routine maintenance should uncover most concerns.

--John Pereira (FSS)

Founded in 1983, Unidata is UOP's oldest program, but most of its software development has taken place in the 1990s. "Our code has all been written within the last seven years, using the newer languages with none of these Y2K problems," reports Sally Bates. Nearly all of Unidata's software is created in house, and "the software we distribute by others has all been made Y2K compliant." For example, the McIDAS interactive display package, created two decades ago at the University of Wisconsin, has reportedly been brought up to speed by UW.

Contingency planning

Velma Ryan (Food Services) has been thinking about contingency planning. "There may be problems with my suppliers' computers. I don't want to take a chance, so I plan on being pretty well stocked at the end of the year," allowing time for suppliers to resolve their problems. If there are power brownouts, she'll have food on hand, like sandwiches, that doesn't require cooking.

If there are power fluctuations, SCD's supercomputers can be safeguarded in a number of ways. The uninterruptible power system (UPS) gives machine-room technicians about 15 minutes to power down, protecting delicate disk drives from power spikes. SCD's Infrastructure Support Group has installed a new UPS that, among other things, is Y2K compliant. (Problems in rebooting equipment after the UPS installation resulted in a computer shutdown that affected several SCD systems on Friday, 22 January.)

A history of cooperation and communication with Public Service Company provides UCAR with another level of protection. "We have an account representative, so we have direct contact," says John Pereira (FSS). After the snowstorm in April 1997 there were successive power outages the next day. "Problems started about seven in the morning," John recalls. Public Service "told us they couldn't guarantee stable power for the next 24 hours or so. So SCD decided not to bring everything back up until it had stabilized. . . . The decision to shut down happened about ten in the morning, and the computer room didn't have stable power until sometime after midnight."

If, as the most extreme Y2K warnings predict, the power grid were to go down for an extended period, UCAR facilities would close until power was restored, according to Steve Dickson. "We've got emergency procedures for closing the facilities, with all the steps and who's responsible. . . . We can essentially mothball our facilities. The chances of that are very remote, but it's the sort of thing we need to talk about."

The part of emergency planning that Steve Dickson would like to see develop further is communication. "The year 2000 problem is going to have at least a community-wide impact, not [just in] individual locations." He would like to see ongoing communication with UCAR constituents "so they understand where we are, what our problems are, and have confidence that we are giving them as much information as we can [and are working] to minimize the inconvenience."

Steve Sadler, who's been in charge of Health, Environment, and Safety, has just donned an additional hat as director of all those functions plus Security. As he settles into his new responsibilities, he'll be carving out some time to investigate Y2K issues and consult with others around the institution. "As far as security goes," says Steve Dickson, "we're interested in data security"--keeping hackers out--"and physical security. We'll be looking at both of those things to make sure that we're on our toes."

According to UCAR president Rick Anthes, "It's important that we all take the year 2000 issues seriously, but at the same time not overreact. UCAR and NCAR management are committed to taking all reasonable actions to make the transition from 1999 to 2000 as smooth and nondisruptive as possible. I am confident that everyone in the organization will help make that happen, so we can get on with our business of research and service."

So how does project leader Steve Dickson sum up UCAR's preparations for New Year's Eve 1999? "We've made an incredible amount of progress, and I'm pleased with that. But the problem is we're running out of time. As the deadline approaches, the attention will increase. But it's about time to turn on the afterburners and hit it. And I think we're doing that, in most areas." •Zhenya Gallon

Staff Notes Monthly will revisit UCAR's Y2K planning as the year progresses.

Y2K resources and answers


In this issue...
Other issues of Staff Notes Monthly


UCAR
NCAR
UOP

Edited by Bob Henson, bhenson@ucar.edu

Prepared for the Web by Jacque Marshall