NIST urges IT quality checks
The government's growing reliance on commercial software puts federal users at risk because of bugs in software that is rushed to market, a National Institute of Standards and Technology study has concluded. What government users need is a better way to measure the quality of the information technology tools they buy, said Mike Hogan, the NIST IT Lab's standards liaison and the study group's leader. The group concluded that NIST should rethink its role in
The government's growing reliance on commercial software puts federal users at risk
because of bugs in software that is rushed to market, a National Institute of Standards
and Technology study has concluded.
What government users need is a better way to measure the quality of the information
technology tools they buy, said Mike Hogan, the NIST IT Lab's standards liaison and the
study group's leader. The group concluded that NIST should rethink its role in rating IT
standards.
"We are measuring and testing programs," Hogan said. "But even if we
develop a timely test tool that everyone wants to use, it does not mean we should do what
we used to and set up a testing program for the government, then run it for a
decade."
NIST formed the group to study how NIST measures IT performance. The team of technical
NIST employees found that common measurement tools used to examine other items did not
work with IT. Though distance can be measured in miles, and steel by its relative
hardness, NIST can't quantify IT in a similar way, Hogan said.
Because computer technology changes so rapidly, NIST does not want to spend months
creating measurement tools for a new language or program group only to find the technology
obsolete once the agency decides on a standard, the group said.
One of the most important things the study group found was the need for a tool to
advise companies and government users when a product has been fully tested, Hogan said.
"Wouldn't it be nice if there was a way people could tell management they could
spend $100 million less on testing," he said. "Right now, people are using the
seat of their pants or their experience to try to determine how much testing is
enough."
For many manufacturers, the lack of standards to rate IT results in embarrassing flaws
such as security holes in World Wide Web browsers or operating systems that crash home
computers, Hogan said. But because some government computers run applications that can't
afford to fail, the need for testing standards is apparent, he said.
"Some of these applications, for example, run the phone system. So they have to
work," Hogan said. "If the air traffic control people can't use their phones,
then it becomes a safety issue."
The NIST study group found five methods being used to test IT tools and programs. None
was without flaws, Hogan said. The most widely used method is conformance testing, where
products are measured against a standard.
The problem is that often no standard exists. Instead, a wildly successful commercial
product becomes the de facto standard against which all similar products are tested, the
report said.
Hogan said the NIST IT Lab should take a more active role in determining standards, but
he cautioned that requests for too many standards could swamp the agency.
Although the study made no formal recommendations, Hogan said the report confirmed that
measuring IT was valid and necessary in software development.
He said he expects the role of the NIST IT Lab to grow, but testing standards need to
be put in place if software bugs are going to become a thing of the past.
IT measurement "will be a key to U.S. competitiveness and international commerce
in the 21st century," the report said. "Supporting the specific priority IT
testing and measurement needs of U.S. industry should be key goals for NIST."
NEXT STORY: Interior Dept. upgrades its procurement system