How Virginia Juked Its COVID-19 Statistics

Combining positive and negative results from the two tests in the same statistic, as Virginia has done, makes no sense.

Combining positive and negative results from the two tests in the same statistic, as Virginia has done, makes no sense. Shutterstock

 

Connecting state and local government leaders

The state is combining results from viral and antibody tests in the same statistic. This threatens to confound America’s understanding of the pandemic.

The United States’ ability to test for the novel coronavirus finally seems to be improving. As recently as late April, the country rarely reported more than 150,000 new test results each day. The U.S. now routinely claims to conduct more than 300,000 tests a day, according to state-level data compiled by the COVID Tracking Project at The Atlantic.

But these rosy numbers may conceal a problem: A lack of federal guidelines has created huge variation in how states are reporting their COVID-19 data and in what kind of data they provide to the public.

These gaps can be used for political advantage. In at least one state, Virginia, senior officials are blending the results of two different types of coronavirus test in order to report a more favorable result to the public. This harms the integrity of the data they use to make decisions, reassure residents, and justify reopening their economies.

Other differences make it hard to track the pandemic. In at least three other states, officials have lumped together probable and confirmed COVID-19 deaths; most don’t specify how they’re counting deaths. While most states report the number of people who have been tested for the coronavirus, six states say they track the number of samples that have been tested—and California and New Jersey switched methods in the past few weeks. Louisiana, Nebraska, Nevada, Vermont, and New York do not report the racial or ethnic breakdown of coronavirus cases. Even more states fail to report the racial breakdown of deaths. It is still impossible to know, for example, how many black people have died of COVID-19, though the data that do exist suggest that black people are dying at much higher levels than other groups.

There are many ways that the lack of data has complicated the outbreak. Until May 12, the Centers for Disease Control and Prevention had not reported state-level testing data, leaving efforts such as our COVID Tracking Project to fill that gap. The lack of reliable national testing data has made it difficult for modelers, policy analysts, and others to understand the true scope of the outbreak. Data about hospitalizations have been even harder to understand. Because the states report hospitalization counts in fundamentally different ways, there is no way to calculate the number of people who have been hospitalized with COVID-19 in the U.S.

Many pandemic response efforts assumed clean, standard, accessible data would exist, but they do not. That said, Virginia’s decision to mix the results of two different kinds of tests marks a new low in data standards.

The state is reporting viral tests and antibody tests in the same figure, even though the two types of test answer different questions about the pandemic and reveal different types of information. By combining these two types of test, the state is able to portray itself as having a more robust infrastructure for tracking and containing the coronavirus than it actually does. It can represent gains in testing that do not exist in reality, says Ashish Jha, the K.T. Li Professor of Global Health at Harvard.

“It is terrible. It messes up everything,” Jha told us. He said that combining the test results, as Virginia has done, produces information that is impossible to interpret.

The state’s decision to combine the tests was first reported by the Richmond Times-Dispatch.

The two tests have little in common. Viral tests help officials do the basic blocking and tackling necessary to contain an outbreak. If someone tests positive on a viral test, they are still infectious, so they can be told to self-isolate in order to protect the susceptible population. Public-health workers can trace their contacts to find others who may be infected with the coronavirus but who are not yet experiencing symptoms. Viral tests can also be used to monitor people who work in high-risk environments—such as a meatpacking plant—to diagnose a contagious person before they spread the disease.

Antibody tests, on the other hand, allow for something closer to post-game analysis. They help officials understand the true number of people in a state or city who have been exposed to the coronavirus. But they do so on a lag: Individuals who test positive on an antibody test are likely no longer infectious, and were infected by the coronavirus at least a week earlier.

The two tests do not even examine the same specimens. Viral tests analyze a throat swab, nasal swab, or saliva sample. They are sometimes called “PCR” tests, after the polymerase-chain-reaction technique used to isolate viral genetic material. But antibody tests use a blood sample. They are sometimes called “serological” tests, because they analyze the blood serum.

In other words, combining positive and negative results from the two tests in the same statistic, as Virginia has done, makes no sense.

But commonwealth officials say they have no choice. Other states are mixing their results, claimed Clark Mercer, the chief of staff to Governor Ralph Northam, at a press conference this week.

“You can’t win” by keeping viral and antibody findings separate in public data, he said, adding that combining the two tests’ results was the only way to improve Virginia’s position in a list of states ranked by the number of tests they had conducted per capita. “If another state is including serological tests, and they’re ranked above Virginia, and we are not, and we’re getting criticized for that, [then], hey, you can’t win either way. Now we are including them, and our ranking will be better, and we’re being criticized,” he said.

We could not find evidence that other states are blending test results in the way that Mercer claimed. In an email, a spokesperson for the Virginia Department of Health claimed that Arizona, West Virginia, and the District of Columbia also mingled viral and antibody results. This is false: Those three governments either separate out, or do not report, the result of negative antibody tests to the public.

Other states report positive serological tests as “probable” COVID-19 cases. This is in line with recommendations published by the Council of State and Territorial Epidemiologists, a nonprofit that works with state and local epidemiologists in the U.S. At least 16 U.S. states and two territories have reported such “probable” cases, although they may not always have done so using serological tests, according to the CDC. However, Arizona, which reports a substantial number of positive serological tests on its own website, does not show up on the CDC’s list. Nor does Kansas, which explicitly states that they are including such probable cases.

While including antibody tests in a state’s total creates too rosy of a testing picture for a state, reporting only probable positive cases without disclosing how many antibody tests are being completed could actually make the situation look more dire in a state than it is.

The spokesperson said that Virginia planned to “disaggregate” its viral and antibody results in the future, but he did not provide a firm date.

Kathy Turner, deputy state epidemiologist for Idaho and the presenting author of the CSTE standards document, did not criticize Virginia’s decision, but she did lay out why her own state decided to keep PCR and serology tests separate.

“[I]n Idaho, we have decided to only display viral tests because those are the denominator we use to calculate our percent positivity rate and we are very confident what they mean,” she told us. “Additionally, we focus on the PCR tests because we can compare the percent positivity over time—before serology tests were available.”

Blending the results also misstates Virginia’s success at improving this crucial metric, sometimes called the “test-positivity rate.” This measurement compares the number of people who have tested positive for the coronavirus to the number of people who have been tested overall. In April, one in five Americans who received PCR tests for the virus were found to be infected, a very high rate that suggested only the sickest people could get a test. For the past week, fewer than one in 10 tests in the U.S. have found a positive result, according to state data. Some of this improvement is certainly the result of the New York metro area’s waning outbreak.

Leaders in many states, including Virginia, have cited the local test-positivity rate to justify loosening shelter-in-place restrictions. Northam has repeatedly said that Virginia’s test-positivity rate had to fall for 14 days before he would loosen restrictions.

But because Virginia combines viral and antibody results, its positivity rate is unusable, said Jha, the Harvard professor. The positivity metric is only useful when describing the result of viral tests, because it is meant to provide a rough estimate of how many people infected with the coronavirus are getting tested for it. Antibody tests, which are meant to sample a broad swath of the healthy population, should not be included in it. By lumping the two tests together, as Virginia has done, states can artificially improve their test-positivity rate.

Only by keeping the two types of test separate can the country—and the commonwealth of Virginia—understand the true scope of its outbreak, experts say.

“You’re comparing apples to pears,” Gigi Gronvall, a senior scholar at the Johns Hopkins Center for Health Security, told us. Viral and antibody tests “look a little bit alike, but it doesn’t let you make the comparison needed. So why not keep them separate?” Gronvall has written about the need to expand antibody testing across the U.S.

She is also worried that the data are statistically meaningless, because viral tests have fewer false-positive errors than antibody tests. “There’s so much variability in the antibody tests that it’s like taking an iffy number and throwing it in with some more reliable numbers,” she said.

Although combining the data from two different tests may seem like a technicality, the decision threatens to confound some of the most important questions about the coronavirus’s path in the United States. How many people are sick right now? How many people can the U.S. actually test for the coronavirus every week? Is the situation outside the New York metropolitan area getting better or worse? Answering these questions requires stable data about how many people have received a diagnostic test and how many of those people are infected.

The scope of the test-mixing problem is not yet clear. No other state aside from Virginia has admitted to counting antibody tests in their overall totals. We do know, however, that large numbers of antibody tests are being completed in the U.S. but not reported to the public by most states. As test numbers have shot up, Quest Diagnostics—one of the two largest commercial laboratories in the country—reported doing only 200,000 PCR tests from May 4 to May 11, which is 180,000 fewer diagnostic tests than in the preceding week. In fact, from May 4 to May 11, the company did almost 100,000 more antibody tests than PCR ones. LabCorp, the other major commercial reference laboratory, has not released similar data, but it has stated that its diagnostic- and antibody-test capacity are about equal.

Two states do report viral and antibody tests separately. In Colorado, 30 percent of tests completed so far in May were for antibodies. In Arizona, 23 percent of the total number of the tests ever done in the state have been serological. In both states, antibody tests started to be conducted en masse around April 26.

This timeline matches up disturbingly well with the improvement of the national picture, which saw a sudden jump at the end of April from an average of about 150,000 tests per day to 200,000, 250,000, and now 300,000 tests per day. The White House has celebrated the improvements in testing, noting in a press conference this week that the United States does more tests per day than any other country. “We have met the moment and we have prevailed,” President Trump said.

In fact, the U.S. has tested a smaller share of its population than other industrialized countries, including Italy, Canada, and Germany.

“I find our testing record nothing to celebrate whatsoever,” Senator Mitt Romney, a Republican of Utah, told Brett Giroir, the assistant secretary for health, at a hearing yesterday. “You celebrated that we had done more tests, and more tests per capita even, than South Korea. But you ignored the fact that they accomplished theirs at the beginning of the outbreak, while we treaded water during February and March.”

It’s possible that Virginia is alone in its reporting methodology, but until we know how many states are dumping antibody tests in their totals, the White House’s claims that the U.S. has overcome its testing plateau cannot be given full weight.

It was one thing to go into the outbreak blind because of the lack of testing, as the U.S. did. It’s another to choose to cloud our vision. Antibody-testing data are an important part of understanding the outbreak; PCR diagnostic-testing data are also an important part of understanding the outbreak. But if states mix the two together, the value of that information plummets.

The good news is that laboratories do report the type of tests they’ve conducted to state governments and the CDC. All states should have that data at their disposal. So should all their residents—and all Americans.

Alexis C. Madrigal is a staff writer at The Atlantic and the author of Powering the Dream: The History and Promise of Green Technology. Robinson Meyer is a staff writer at The Atlantic, where he covers climate change and technology.

NEXT STORY: Millions Stuck At Home With No Plumbing, Kitchen Or Space To Stay Safe

X
This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising and analytics partners. Learn More / Do Not Sell My Personal Information
Accept Cookies
X
Cookie Preferences Cookie List

Do Not Sell My Personal Information

When you visit our website, we store cookies on your browser to collect information. The information collected might relate to you, your preferences or your device, and is mostly used to make the site work as you expect it to and to provide a more personalized web experience. However, you can choose not to allow certain types of cookies, which may impact your experience of the site and the services we are able to offer. Click on the different category headings to find out more and change our default settings according to your preference. You cannot opt-out of our First Party Strictly Necessary Cookies as they are deployed in order to ensure the proper functioning of our website (such as prompting the cookie banner and remembering your settings, to log into your account, to redirect you when you log out, etc.). For more information about the First and Third Party Cookies used please follow this link.

Allow All Cookies

Manage Consent Preferences

Strictly Necessary Cookies - Always Active

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data, Targeting & Social Media Cookies

Under the California Consumer Privacy Act, you have the right to opt-out of the sale of your personal information to third parties. These cookies collect information for analytics and to personalize your experience with targeted ads. You may exercise your right to opt out of the sale of personal information by using this toggle switch. If you opt out we will not be able to offer you personalised ads and will not hand over your personal information to any third parties. Additionally, you may contact our legal department for further clarification about your rights as a California consumer by using this Exercise My Rights link

If you have enabled privacy controls on your browser (such as a plugin), we have to take that as a valid request to opt-out. Therefore we would not be able to track your activity through the web. This may affect our ability to personalize ads according to your preferences.

Targeting cookies may be set through our site by our advertising partners. They may be used by those companies to build a profile of your interests and show you relevant adverts on other sites. They do not store directly personal information, but are based on uniquely identifying your browser and internet device. If you do not allow these cookies, you will experience less targeted advertising.

Social media cookies are set by a range of social media services that we have added to the site to enable you to share our content with your friends and networks. They are capable of tracking your browser across other sites and building up a profile of your interests. This may impact the content and messages you see on other websites you visit. If you do not allow these cookies you may not be able to use or see these sharing tools.

If you want to opt out of all of our lead reports and lists, please submit a privacy request at our Do Not Sell page.

Save Settings
Cookie Preferences Cookie List

Cookie List

A cookie is a small piece of data (text file) that a website – when visited by a user – asks your browser to store on your device in order to remember information about you, such as your language preference or login information. Those cookies are set by us and called first-party cookies. We also use third-party cookies – which are cookies from a domain different than the domain of the website you are visiting – for our advertising and marketing efforts. More specifically, we use cookies and other tracking technologies for the following purposes:

Strictly Necessary Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Functional Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Performance Cookies

We do not allow you to opt-out of our certain cookies, as they are necessary to ensure the proper functioning of our website (such as prompting our cookie banner and remembering your privacy choices) and/or to monitor site performance. These cookies are not used in a way that constitutes a “sale” of your data under the CCPA. You can set your browser to block or alert you about these cookies, but some parts of the site will not work as intended if you do so. You can usually find these settings in the Options or Preferences menu of your browser. Visit www.allaboutcookies.org to learn more.

Sale of Personal Data

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Social Media Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.

Targeting Cookies

We also use cookies to personalize your experience on our websites, including by determining the most relevant content and advertisements to show you, and to monitor site traffic and performance, so that we may improve our websites and your experience. You may opt out of our use of such cookies (and the associated “sale” of your Personal Information) by using this toggle switch. You will still see some advertising, regardless of your selection. Because we do not track you across different devices, browsers and GEMG properties, your selection will take effect only on this browser, this device and this website.