Tera Rex: How supercomputing is roaring back

By Susan M. MenkeGCN StaffDuring supercomputing's first heyday in the late 1970s, the federal government spent millions on big iron from a stellar array of hardware makers. The giant systems of that era solved huge sets of equations that turned into global weather models, gene maps and nuclear weapon simulations. But gradually the hot-running vector machines grew too big and costly for their niche.Most of the hardware stars born then'Alliant Computer Systems, BBN Corp., Cray Computer Corp., Cray Research Inc., Fujitsu Corp., Intel Supercomputer Systems, Hitachi Data Systems Corp., Kendall Square Research, MasPar Computer Corp., Multiflow Computer Corp., nCube Corp. and Thinking Machines Corp.'have dwindled or gone out of the supercomputing business.'It was the great extinction,' said Thomas Sterling, senior scientist at NASA's Jet Propulsion Laboratory. 'There were no new, good ideas.'Of the group, only Cray Research, now owned by SGI, is still aggressively marketing new supercomputers. And market latecomer IBM Corp. has made a remarkable leap from its mainframe history to join in supercomputing's comeback.The vector uniprocessor, pioneered by Seymour Cray in the 1970s, executed consecutive instructions on one-dimensional data arrays. Bottlenecks in the silicon circuits eventually forced Cray to try making faster semiconductors out of toxic gallium arsenide'without success. Vector supercomputing stalled.The mid-1980s saw attempts by others to execute many instructions at once by threading streams of data through multiple processors. Hardware makers headed off in different directions, toward parallel vector processing, scalable parallel processing, massively parallel processing and symmetric multiprocessing.Lead by designThey designed mini-supercomputers and super-minicomputers. They clashed over the efficiency of single-instruction-multiple-data streams vs. multiple-instruction-multiple-data streams vs. very long instruction words.Despite the difficulty of interconnecting dozens or hundreds of CPUs and programming software to exploit them, the new multiprocessor machines began to overtake vector performance.Soon the race was on to build proprietary processors, coprocessors, operating system extensions, interfaces and protocols that could eke out a bit more performance.Along the way, a few government scientists had an epiphany: Why not emulate parallel construction on a macro scale? They simply hooked together cheap, common PCs as computing nodes with Ethernet, Fast Ethernet and, more recently, Gigabit Ethernet connections.The Jet Propulsion Laboratory's Sterling, considered the father of the Beowulf class of commodity supercomputers, said the massively parallel clusters cobbled together at government labs in the early 1990s 'established a low level of expectation' compared with commercial vendors' behemoths. But Beowulf construction was much, much cheaper.'It was a critical-mass convergence of market trends,' Sterling said. 'We wanted high-end computers for a broad range of problems. We wanted user-driven, vendor-independent, straightforward, easy, incremental [systems] from ideas already applied elsewhere.'And a new race was on. Tall racks of interlinked commodity PCs, some casually named LOBOS for lots of boxes on shelves, grew up in NASA and Energy Department laboratories as well as at other federal sites. Researchers could string them together and unhook them as needed.The crucial choice of a Beowulf operating system'freeware Linux installed on each node'met the federal researchers' need for open source code they could modify to fit their experiments. Coincidentally, it set off an OS revolution of its own.Hardware vendors took note of what the users were doing. Leading makers such as IBM, Hewlett-Packard Co., SGI and Sun Microsystems Inc. began building servers with dozens, hundreds, even thousands of processors running various Unix OSes. They tinkered with shared and distributed memory, plus clustering software to boost their boxes' performance far beyond what the homegrown commodity clusters could deliver.Even for common desktop PC processors, symmetric multiprocessing started to reach toward the eight-way level. Intel Corp.'s crossbar interconnect switch, for example, opens up simultaneous access to eight commodity processors, their memory and input/output components'a feat formerly possible only at near-supercomputer prices.Larry Smarr, director of the National Computational Science Alliance at the National Center for Supercomputing Applications in Urbana, Ill., has called this surge the third wave of supercomputing, supplanting the vector systems of the 1970s and the scalable Unix systems of the late 1980s and early 1990s.A supercluster'the term is more familiar to astronomers than to computer users'need not have all its components or its users physically close together. The supercluster's users can do ordinary tasks at their desktop systems while also drawing huge amounts of transparently homogeneous power and storage from the other systems, interconnected by Fast or Gigabit Ethernet.Jack Dongarra of Energy's Oak Ridge National Laboratory in Tennessee has termed it metacomputing. He envisions users working at simple PC clients but accessing world-class applications and machines on multiple networks via agent software.A supercluster of Defense Department and federally funded research sites calling itself the National Technology Grid already reaches all the way across the United States to the Maui High-Performance Computing Center in Hawaii. A typical site on the grid has several hundred Pentium II processors in nodes running Linux, Microsoft Windows NT or proprietary Unix.In the meantime, supercomputers themselves are roaring back into history.Under Energy's Accelerated Strategic Computing Initiative, the government is sponsoring development of supercomputers that by 2004 will execute 100 trillion floating-point operations per second. The 100-TFLOPS goal, consuming an estimated half-billion federal dollars per year, represents almost a hundredfold quantum leap beyond current supercomputers, however.By at least one leading measure, the fastest federal machine at present is at Energy's Sandia National Laboratories in Albuquerque, N.M.: the 2.12-TFLOPS ASCI Red, built from almost 10,000 Pentium Pro processors.'' Last November and again in June, ASCI Red was designated the world's top-performing supercomputer on the Linpack benchmark, according to the Mannheim Univers ty Web site, at www.top500.org.Linpack, designed by Oak Ridge's Dongarra, measures GFLOPS performance at solving a dense system of linear equations instead of counting processors or applying other metrics.Federal supercomputers dominated the www.top500.org list of the world's top 500 supercomputers, which the university released this month. The U.S. government funds or owns 99 of them; 18 are run by classified programs.Ranking orderLos Alamos National Laboratory's ASCI Blue Mountain and Lawrence Livermore National Laboratory's ASCI Blue Pacific Silver rank in the top 11. SGI systems at the Naval Oceanographic Office and NASA's Goddard Space Flight Center are halfway to the 1-TFLOPS range.Farther down the list are two of the government's own homegrown commodity supercomputers.Sandia's 54.2-GFLOPS Cplant cluster, described by the lab as growing like a pumpkin vine, will eventually extend its tendrils into Energy's enormous ASCI infrastructure. It ranks No. 129 and is considered the world's fastest Linux machine.The second homegrown government winner is the 48.6-GFLOPS Avalon cluster at Los Alamos. Ranked No. 160, the $400,000 cluster consists of Alpha processors with Fast Ethernet interconnections.And supercomputer evolution continues. NASA's Ames Research Center in California, for example, is heading up research into petaFLOPS, or 1,000 TFLOPS, computing. The center will design computing architectures based on molecular nanotechnology. The goal of nanotechnology is precise siting of individual atoms within circuits and materials by about 2008.Ten years from now, a look back at the current feverish supercomputing efforts might make the government's research labs seem like the curators at the Jurassic Park of computing.