A Staff Notes Monthly special report on UCAR and Y2K appeared in the February issue. For this update, we revisited some staff and added a few others to get a sense of where things stand in the late fall of 1999.
Red, yellow, and green traffic lights showed members of the UCAR Management Committee how far each division and program has come along the road to Y2K compliance during presentations at the 16 September and 21 October UMC meetings. There were red lights in only 8% of 77 listed tasks as of October, and none of those pertained to mission-critical functions. (Monthly updates can be tracked internally on the Web.)
GST and the week-zero problemMany folks had their eyes on the GPS rollover problem this past summer, seeing it as a dry run for New Year's Eve. So how did things go on 21 August when the ten-bit binary code in GPS receivers rolled over to 0000000000? "There were no big disasters that I heard about," reports GST's Stick Ware. There were scattered reports from around the world of some receivers failing, but none that had a serious impact on safety or well being.
Lou Estey kept track of GST's own week-zero experience. "There were a few glitches, but surprisingly few." Since time is the key element in GPS, receivers without updated firmware did not function properly. (Firmware is programming that's created and distributed like software but is inserted into programmable read-only memory where it then becomes a permanent part of the programmed device.) This was the case with a dormant receiver on GST's prototype testing grounds at NOAA's Platteville site, east of Longmont. "When we went out there [in late September] it was very confused. As soon as we installed the new firmware it was fine."
As for GST's preparedness for the year 2000, Stick says, "Everybody reports they're compliant and ready. We don't have any public safety issues, so if a data stream [from a continuous site] stops, it's not going to be a disaster."
Lou described his thinking on preparations for New Year's Eve. "In the U.S. there might be a few glitches, but I think they're going to be hammered out within hours to days for the major things. Around the world I think it's going to depend on what nation you're talking about. I'm not going to lose sleep over it. I'm going to make sure my Coleman stove has enough fuel for a couple of days just in case there's a disruption around here, but I'm not anticipating it." (ZG
There's another factor keeping systems in the yellow category: "The longer you wait, the better upgrade you get [from vendors]," says Steve. That's a refrain repeated by staff in each group interviewed for this update.
Once Steve pulled together the September and October reports, he says, "I was really impressed. I'm gratified at what good shape we're in at this time." As reported in February, UCAR's Y2K Task Group identified a small number of functions essential to the UCAR and NCAR mission: health and safety, scientific supercomputing and its support services, networking and communication, and payroll and other financial services. Steve's traffic-light reports include these mission-critical systems and others considered secondary. "It's virtually certain that all our mission-critical systems--and most noncritical ones--will operate on January first."
When asked whether any good had come out of our Y2K preparations, Steve suggested that keeping systems up to date and achieving uniformity of operating systems within groups was useful. "We've upgraded a lot of software. And having to do these inventories gives the systems administrators a data base they can use to continue to keep things current."
Summing up, Steve says, "We're right where we expected to be when we drafted our plan two years ago. [Even] with all of the things that we didn't know about then, we got to September 30th, 1999, having accomplished essentially what we set out to accomplish." Upgrading and shakedown testing will continue throughout the organization through the end of the year.
As noted in February, all of the supercomputers have been Y2K compliant since the beginning of the year. Testing of front-end systems continued through the summer. That effort revealed some deficiencies: for example, the workstations that boot the Cray machines have software that requires older versions of Sun's operating system. But "any issues we've run across are ones that we can get around by changing our operational procedures," says Tom. "We'd rather do that than install an upgrade that might destabilize the entire environment." So while the SCD inventory lists those workstations as "not Y2K compatible," they are ready to run, with human assistance, into the next century. A few other noncompliant systems will be decommissioned by the end of the year.
August was a busy month in SCD. To simulate the way computing jobs are actually done and to test multiple vendor interactions, HPS staff created a multiplatform, heterogeneous test environment including the Network Engineering and Technology Section's NETS testbed. some Sun and SGI servers, and the test partition of the NCAR Mass Storage System. Then the team rolled the clock forward through New Year's Eve, on to leap day in February, and further into 2000. "We found a couple of minor 'gotchas' in log files"--two-digit date entries--"and those got corrected. Outside of that, the testing didn't expose any problems," Tom reports. Also in August, the new IBM SP (blackforest) arrived--with a Y2K-compliant operating system already installed.
Commenting on all the activity, division director Al Kellie says, "I'm very pleased with the high degree of professionalism that SCD has shown in its preparations for the year 2000 and the leadership that Tom Engel has demonstrated on behalf of SCD."
HPS continues to test individual platforms and to apply patches as vendors release new ones. "I anticipate that this will be an ongoing process up through the end of the year and, frankly, into next year," Tom explains.
SCD's plans for New Year's weekend are "like planning for a snowstorm," according to Tom. Production systems are scheduled to run through the weekend. The computer production group will have additional staff on duty around the clock. HPS staff will provide additional monitoring: Off-duty engineers and production staff will be wearing pagers, "and if the phone system goes down, we'll all jump in our cars."
Paychecks, travel reimbursement, and all that"We've completed our year-2000 testing and all looks well with our financial systems," says Steve Hinson, manager of Applications and Development in the Information Technology (IT) section of F&A. Bi-Tech handles every financial transaction at UCAR, including payroll, general ledger, purchasing, accounts payable, and accounts receivable. "Everything within Bi-Tech has been tested and will work in the year 2000."
Steve and his team put together a free-standing clone of the accounting system and ran it into the year 2000. "We never really thought there would be a major issue with the software, and it was fine." While testing has been ongoing throughout the year, an intensive period began in early September and ran for six weeks. A total of 13 staff from IT, Contracts, General Accounting, and Finance were involved, but "their day-to-day jobs were not put on hold as a result of the testing," Steve says. "We took the same practical, common-sense approach that the UCAR Y2K Task Group identified. It's monotonous and laborious, but it's something that needed to be done, so we just did it," he explains, adding, "I'm just glad I won't have to deal with the year 3000." ZG
Susan also consulted ATD's engineers on the wide variety of instruments the division is known for. "I concluded that any problems would be pretty minor, and we would deal with those as they happened." Problems like an incorrect date turning up on a plot can be worked around. "It's not going to crash the system. The year doesn't have a big impact on the operation of the instruments."
Ron Ruth and Chris Webster have been riding herd on data systems for RAF, and both are confident that a new data acquisition system coming on line in November will handle the new year quite well. "Our processor takes care of the date when it gets there. Everything downstream is a four-digit integer, not a character string," Chris explains.
The avionics and onboard instrumentation on RAF's Electra and C-130 have been vetted, and division director Dave Carlson is confident everything is functional. But no flights and no field projects are scheduled for December or January. "This is the first time in at least five years that we haven't had any projects during that period. We don't have anything that needs to work during the rollover," Dave says.
Also planned for early November is an MMM testbed. "We're taking one machine from every platform and rolling it to 2000. Then we'll connect it to the NETS testbed, and people will be able to test all their applications to see if there are any problems."
So far, Pat, Bill Boyd, Jody Williams, and José Castilleja have been able to fit Y2K preparations into their normal schedule without major disruption. A change is in store for New Year's weekend, however. "We're shutting all our systems down on the 31st and probably coming in that Sunday to start bringing things back up, depending on what's happening with power."
Consumer tips for Y2KDesktop systems: Several computer-savvy staff interviewed for this update recommended powering off your desktop computers before you take off for the New Year's holiday. There are two reasons, both related to potential fluctuations in electrical power from our local utility company. First, it puts wear and tear on electronic components to start and stop them frequently. Second, if you're attached to a network, your systems administrator will thank you for not putting a load on the servers when several remote systems try to reboot all at once.
VCRs: Older VCRs will not know what date to use in their programmed recording feature after 12/31/99. A simple workaround is to set your VCR for the year 1972, whose days of the week correspond to those in 2000. ZG
Trane, the manufacturer of the chillers that cool the SCD computer room as well as the rest of the Mesa and Foothills Labs, has verified that all but two of the six chillers are Y2K compliant. Serial numbers and manufacturing records are being checked for the final two chillers as of this writing.
John's approach has been to keep after the vendors to be sure they deliver what they've promised on schedule. Because the vendors have nonrenewal clauses in their contracts with us, "it gives us a little leverage. Most of them are pretty good as far as customer response is concerned."
Both the telephone system and the card-reader system that unlocks doors to all our buildings have received software upgrades. Teresa Shibao of Telecommunications and Security Systems reports that evaluation has been going well, with plans for all tests to be completed by 12 November.