In a test of online sites aimed at government buyers, a team of reviewers finds more to dislike than to like

By Richard W. WalkerGCN StaffElectronic-commerce Web sites for government buyers are popping up everywhere. A few of them have been around for a while, though many measure their histories in months or even weeks. More are likely to follow.How easy are these sites to use? How are their search engines? Do you have to be registered, or can you window shop? What happens when a first-time government visitor tries to buy something on one? To find out, GCN set up a bank of PCs in a conference room and asked four procurement specialists and one Web expert to surf a sampling of government-run and commercial Internet marketplaces and evaluate them according to a range of criteria.Our site evaluators were Jack Braun, a contract cost analyst for the General Services Administration's Federal Technology Service; Gabriel Goldberg, an independent consultant and Web expert from Alexandria, Va.; Tiffany Hixson, a contracting officer in the Office of Acquisition Management in the Commerce Department's Office of the Secretary; Chip Mather, senior vice president of Acquisition Solutions Inc. of Chantilly, Va., and a former Air Force procurement executive; and David Ramos, acting director of logistics and acquisitions operations at the National Institutes of Health in Rockville, Md.The sites they examined included the Air Force Information Technology Superstore, at , and Federal Supply Service's GSA Advantage, at . The commercial sites were , developed by eFederal Inc. of Aliso Viejo, Calif.; , developed by World Wide Technology Inc. of St. Louis; , developed by Digital Commerce Inc. of Herndon, Va., and of Goleta, Calif.Only Hixson had used any of the sites under evaluation. She had bought some office products on GSA Advantage. Otherwise, the GCN surfers were first-time users of these sites.All of them, however, had bought computer products from commercial or vendor Web sites, so they were used to buying online.The methodology was fairly simple. They would log on to the sites, look around, try out key features such as help and search, and then go through the processing of searching for and selecting an item or two to buy.The GCN site evaluators did not have an unlimited amount of time to check out each site, even though experience at a site does increase familiarity with its features. But they agreed that online buyers, even first-time ones, should be able to go to a site, quickly find what they're looking for, order it and get out. That's the whole point of buying online.But that simple scenario did not happen for our evaluators. Their experiences were generally disappointing. They encountered problems with finding products, getting detailed information about the products and even registering with some of the sites.'Some of the sites have a long way to go,' Braun said. 'For others it's a matter of refinement. There's an amazing variation as to the quality of what's out there.''The private sector, the business-to-consumer sites, are much further ahead of the business-to-government sites or even the government-to-government sites in terms of sophistication, ease of use and breadth of selection,' Ramos said.'Part of the problem is that [commercial] sites have set expectations that were not met here,' Mather added.Hixson said the sites promised more than they could deliver.'If they were a little more honest about what it is that they're doing, you'd be less frustrated when you're going through them,' she said. 'I think we have a long way to go.'Goldberg despaired of name confusion with sites such as Fedbuy and eFederal. How does a government buyer keep them all straight?Overall, the two government sites, GSA Advantage and the Air Force IT Superstore, got the highest marks.'These guys get it,' Mather said about Air Force Superstore.The GCN evaluators gave both the Air Force and GSA Advantage sites bonus points for offering comparison shopping for computer products.They especially liked the Air Force Superstore's detailed, side-by-side comparisons of prices and product specifications for desktop and notebook PCs and servers.The Air Force site is 'taking it from a buyer's perspective'the person who's trying to order a computer system under federal rules,' Mather said.GSA Advantage also offered a 'nice comparison of prices from multiple vendors at one time,' Braun said.As the GCN team moved through the sites, problems, glitches and general confusion abounded.For instance, GovPlace insisted on showing Ramos a price of $1.08 for a 200-MHz Armada notebook PC from Compaq Computer Corp. 'This is discount computers to the max,' he quipped. 'They've got some errors in their programming.' But errors did not extend to closing the sale this time. After drilling down a few pages, Ramos found that there were none available.Braun ran into other difficulties. Searching for technical details about a monitor on eFederal, he clicked on the product description and came up with 'no results.' Finally, after several tries, the data appeared.'They've got some bugs in their pages,' he said. 'They need to do some work with their links.'Even the generally praised Air Force IT Superstore offered its share of tribulations.Although our evaluators liked the Superstore's ability to show comparisons of computer products, they were confused by the unexplained three-digit numbers used to identify items on the list.'When they listed a group of computers, it started from 100 and went down through 900,' Hixson said. 'What does that mean? Contractor line items?'The numbers, they eventually discovered, were processor speeds.Search functions on the sites generally were inadequate and inconsistent'a huge bone of contention for our site evaluators.'If you don't know how a site is set up, to come in cold and be able to get the information you need is important,' Braun said.The consensus of opinion among the GCN surfers: If you can't find what you're looking for, why bother?Goldberg, for example, couldn't find 19-inch monitors from NEC America Inc. of Melville, N.Y., on Fedbuy.Oddly, a search for the word 'monitors' proved fruitless. He finally found the monitors under 'video products.''It's there, but I had to puzzle it out,' he said.Ramos came up empty searching for a cordless mouse from Logitech Inc. of Fremont, Calif., on GSA Advantage, despite several attempts.'If you did a search for a wireless mouse, it came up with wireless mouse controllers and controller cards,' he said. 'In the industry it's referred to as a cordless mouse. But if you do a search on cordless mice you come up with nothing. Logitech is the big manufacturer of [cordless mice], and I even tried to search for Logitech.'The initial look and feel of a site also was crucial to the GCN site surfers. For them, you can judge a book by its cover when it comes to Web marketplace pages.'Look and feel shape the first, very important, impression,' Goldberg said.He noted, for example, that GovPlace was 'a little cluttered. It's not really clear exactly how to start.'In contrast, eFederal had 'an excellent look and feel,' Goldberg said.Finally, registration was another major peeve.GSA Advantage was one site that let our users look around and search for products without having to register.Others buying sites, including GovPlace, required them to register before doing anything.A delay in confirming registration with FedCenter initially prevented participants from gaining access. A company spokeswoman said FedCenter uses the confirmation process so that it can ensure it makes discounts available to qualified government buyers only. FedCenter accelerated the process but, because of the delay, only two participants reviewed the site.'I think having to log in to browse the catalog is probably a deterrent to most first-time users,' Hixson said. 'If they can go in and see how easy the site is to use, those sites will be more likely to have more users.'