NWS preps for storm of data
Last year, the Cray C-90 supercomputer at the National Weather Service's National Centers for Environmental Prediction produced about 10G of data per day. This fall, when a newly installed 786-processor RS/6000 SP supercomputer from IBM Corp. is upgraded, output will run about 100G per day.
By William JacksonGCN StaffLast year, the Cray C-90 supercomputer at the National Weather Service's National Centers for Environmental Prediction produced about 10G of data per day. This fall, when a newly installed 786-processor RS/6000 SP supercomputer from IBM Corp. is upgraded, output will run about 100G per day.'The amount of data we produce will just keep going up exponentially,' said Art Wick, head of the networking and communications branch.So far, the 155-Mbps OC-3 asynchronous transfer mode links are adequate to feed the supercomputer at the Commerce Department's Census computer center in Bowie, Md., said Carl P. Staton, central operations director. An upgrade to the 622-Mbps OC-12 rate probably lies ahead.'The increased demand of the high-resolution models we are just now beginning to put on the IBM [supercomputer] will require us to increase bandwidth,' Staton said.The supercomputer runs weather models, crunching data from ground sensor and satellite observations to produce numerical predictions for forecasters. The accuracy of predictions depends partly on the degree of resolution the models can achieve.The centers' new SP supercomputer is five times faster than the Cray it replaced and produces weather models with a resolution of 32 kilometers. After the autumn upgrade, resolution will improve to 22 kilometers, and within five years it will be 10 kilometers.Each doubling of resolution means a 16-fold increase in data volume, Staton said.The Cray C-90 was housed in a federal office building in Suitland, Md., with a pair of Cray J-90 computers. The J-90s preprocessed data going into the C-90 and did post-processing of its output. The new IBM eventually will assume all modeling functions, but for the time being the J-90s still massage the raw data and the IBM's output.When the IBM RS/6000 SP was installed in March 1999, data moved too slowly between it and the J-90s, Staton said. The cabling infrastructure of the 50-year-old Suitland building, originally designed as a hospital, was inadequate to serve such high-powered machines, so the new IBM moved to its Bowie home in October.The SP communicates with the J-90s in Suitland over an ATM link provided by Bell Atlantic Corp. Weather offices in Camp Springs, Md., where the modelers work, also have a link to Bell Atlantic's ATM network.'With current model resolution, we are getting by with transfer rates between 3M and 4M per second,' Staton said. Peak use takes about 40 percent of the links' OC-3 capacity, and 'that's OK for us right now.'Data flows from the NWS Office of Systems Operations in Silver Spring, Md., to the Suitland computers over redundant, commercial 10-Mbps fiber-optic lines. The finished models, after being processed by the J-90s in Suitland, return over the same link to Silver Spring for distribution to forecasters at NWS and elsewhere.A second, now inactive, link connects Silver Spring with the IBM SP in Bowie. When the modeling functions now shared by three machines are consolidated onto the IBM, the bandwidth choke point will shift to the 10-Mbps link between Bowie and Silver Spring.No one knows yet how much more bandwidth will be needed and when. It depends on how much data the modelers demand as well as how much is being produced.The centers will continue turning out more and more detailed models, Wick said, but 'we can't be in a mode of sending everything to everybody all the time.'For now, 10 Mbps is adequate, he said. Eventually the centers hope to simply host the model data and let users themselves download what they need. That might be possible with an Internet or intranet site, Wick said, but funding for such a project is unavailable now.A more immediate concern is ensuring uptime of the link to the IBM supercomputer. It currently has an OC-3 backup link connected to the same Bell Atlantic substation as the primary link.'We'd like to make that a full diverse path' by routing the backup to a different substation by the end of the fiscal year, Wick said. Timing will depend on funding.
Supercomputer will provide data faster than weather service can move it
Up, up and away
 | The National Centers for Environmental Prediction may have to upgrade its network to the 622'Mbps OC'12 rate to accommodate increased output from a new supercomputer. |
Up, up and away
NEXT STORY: THE VIEW FROM INSIDE: Walter R. Houser