Antibody Tests For Coronavirus Can Miss The Mark

Apr 14, 2020
Originally published on April 15, 2020 5:02 am

Dozens of blood tests are rapidly coming on the market to identify people who have been exposed to the coronavirus by checking for antibodies against it.

The Food and Drug Administration doesn't set standards for these kinds of tests, but even those that meet the government's informal standard may produce many false answers and provide false assurances. The imperfect results could be a big disappointment to people who are looking toward these tests to help them return to something resembling a normal life.

First of all, it's not clear whether someone who has antibodies to the coronavirus in their blood is actually immune. Your body produces these antibodies within about a week of infection.

In many other diseases, people do have a period of immunity after they have been exposed to a microbe and recover from illness. But that has not been demonstrated yet with the coronavirus.

Another problem is that test results are wrong much more frequently than you might expect. While tests may truthfully say they are more than 90% accurate, in practical use they can often perform far below that level.

Deborah VanderGaast of Tipton, Iowa, would love to know the results of an antibody test.

"What if we're already immune and we just don't know it and we don't have to be scared?" she asks. VanderGaast runs a day care center for children with developmental and behavioral disabilities. They're a lot like little kids everywhere.

"We laugh about the sanitizing everything, because, it's just reinfected just two seconds later," she says with a laugh.

VanderGaast is eagerly awaiting the rollout of a blood test that would be able to tell if she and her staff have antibodies to the coronavirus. They are now hypervigilant, and stay home for 14 days if they were even potentially exposed to the coronavirus. They wouldn't have to do that if they knew they were immune, she says.

VanderGaast says the test isn't available in her county now. But it is starting to take off nationally.

Dr. Jeremy Gabrysch runs a mobile medical service in Austin, Texas. He got a supply of antibody tests made by a major Chinese manufacturer and says he has tested several hundred people in the last few days.

"We offer the test for people who may have suspected they might have had coronavirus back in February or March when testing with the nasal swab [and PCR diagnostic test] was very limited," he says. The charge: $49 a test.

Gabrysch says he only tests people when he has other evidence they might have been exposed. "If they had an illness that sounds like it could have been coronavirus and they had a positive antibody test, then it's very likely that this is a true positive, that they indeed had COVID-19," he says.

The test he's using, produced by Guangzhou Wondfo Biotech in China, boasts a specificity of 99%, which means it only falsely says a blood sample contains antibodies against the coronavirus 1% of the time. But despite that impressive statistic, a test like that is not 99% correct, and in fact in some circumstances could be much worse.

That's because of this counterintuitive fact: The validity of a test depends not only on the technology, but how common the disease is in the population you're sampling.

"It is kind of a strange thing," admits Dr. H. Gilbert Welch, a scientist at Brigham and Women's Hospital in Boston who studies issues surrounding tests and screening. "An antibody test is much more likely to be wrong in a population with very little COVID exposure."

This is a result of statistics, rather than the technology of any given test.

Here's a simple way to look at it. Say you are running a test that gives five falsely positive results in a group of 100 people. That doesn't sound too bad. But consider this. If 5% of those 100 people were actually infected with the coronavirus, you should get five correct test results (true positives), along with the five false positive results.

While the manufacturer can rightly claim the test is 95% specific, in this population "the test will be wrong half the time," Welch says. "Half the people will be falsely reassured."

And test results can be considerably worse.

The Food and Drug Administration does not regulate these tests, but White House coronavirus task force coordinator Dr. Deborah Birx has said that she expects manufacturers to achieve a standard of 90% specificity (and 90% sensitivity, another measure of test performance that's less important in this context).

Here's what would happen if you used a test with 90% specificity in a population in which only 1% of the people have coronavirus. Nobody knows for sure, but that could be the situation in many parts of the country.

In that instance, more than 90% of the positive results would be false positives, and falsely reassuring. (You can run your own examples on the calculator on this page).

Loading...

Tests being put to use now vary widely in their specificity.

Lab giant Becton Dickinson says its coronavirus antibody test has a specificity of 91%. Emory University says its test has a specificity of 95%. Stanford University says its test demonstrated 100% specificity, at least in a preliminary study involving 100 samples. The Centers for Disease Control and Prevention has been developing an antibody test since the early days of the epidemic, but has not yet published results about its performance.

One way to limit the problem of false-positive results is to focus on populations where the disease is more common — in other words, in situations where true positives are much more frequent than false positives. That's likely to be the case in hospitals that have cared for coronavirus patients.

"It would be wonderful for health care workers to know their immune status, just to give them peace of mind," says Dr. Jordan Laser, a pathologist at Northwell Health on Long Island, New York.

Even so, Laser says it would still be a mistake to rely on these results.

"Definitely don't use these tests to change your practices in terms of use of personal protective equipment," he would advise his colleagues. "Definitely do not become comfortable in doing your job in taking care of COVID patients. It really would be more of a psychological benefit."

Jeremy Gabrysch, the doctor in Austin, is also using these tests to satisfy the curiosity of his patients, not to provide actionable guidance about whether it's safe for people to assume they are immune.

"We don't recommend that people interpret the result that way, and we counsel against that," he says. "It's not a get-out-of-jail-free card."

Because of these significant limitations, antibody tests fall short of being the imagined passport that would allow people to get back to business as usual and ignore the coronavirus.

But high-quality antibody tests will still be valuable in instances where individual false-positive results matter less. That would be the case for surveys of entire populations, where errors can be factored out.

In these studies, antibody tests are being used to answer critical questions about where the coronavirus is and how prevalent it is. That information can help officials plan how to let normal lives resume — but across populations as a whole, rather than one person at a time.

You can reach NPR Science Correspondent Richard Harris at rharris@npr.org.

A conversation with California biologist Andrew Cohen inspired this line of inquiry.

Copyright 2020 NPR. To see more, visit https://www.npr.org.

STEVE INSKEEP, HOST:

Blood tests to detect past exposure to coronavirus are starting to hit the market. There are dozens of manufacturers, and the tests do not need FDA approval. So they're going right out the door. But even tests that claim to be more than 90% accurate apparently are not. NPR science correspondent Richard Harris is on the line. Richard, good morning.

RICHARD HARRIS, BYLINE: Good morning, Steve.

INSKEEP: What are the tests supposed to do?

HARRIS: Well, the tests cannot be used to diagnose the disease. Instead, they identify antibodies that appear in your blood about a week after you've been infected. Scientists do not know, though, whether people with antibodies are definitively protected from the disease and, if so, for how long. But that hope - that prospect - is really driving a lot of this excitement. So for example, I talked to Deborah VanderGaast in Tipton, Iowa. She runs a daycare center for children with developmental and behavioral disabilities. They're a lot like little kids everywhere.

DEBORAH VANDERGAAST: We laugh about, you know, the sanitizing everything because, you know, it's reinfected just a - just two seconds later.

HARRIS: VanderGaast is eagerly awaiting the rollout of the blood test in her county. She thinks about her staff who are being hypervigilant not to spread the disease.

VANDERGAAST: If some of the people have already been established to have antibodies, they wouldn't have to go home and isolate for two weeks; they could continue working.

HARRIS: Not to mention peace of mind a test result would bring.

VANDERGAAST: What if we're already immune and we just don't know it and we don't have to be scared?

HARRIS: She says the test isn't available in her area. But it is starting to take off nationally.

Dr. Jeremy Gabrysch runs a mobile medical service in Austin, Texas. He says he got a supply of antibody tests made by a major Chinese manufacturer and he's already run a few hundred tests in the last few days.

JEREMY GABRYSCH: We, you know, offer the tests for people who may have suspected that they had coronavirus back in February or March, when testing with a nasal swab PCR was very limited.

HARRIS: Gabrysch says he only tests people when he has other evidence that they might have been exposed.

GABRYSCH: If they had an illness that sounds like it could have been coronavirus and they have a positive antibody test, then it's very likely that this is a - what we call a true positive - that they indeed had COVID-19.

HARRIS: The test he's using boasts a specificity of 99%, which means it only falsely says a blood sample has antibodies when it doesn't just 1% of the time. But despite that impressive statistic, a test like this is not 99% correct and, in fact, in some circumstances could be much, much worse. That's because of this counterintuitive fact. The validity of a test depends not only on the test itself but, oddly, on how common the disease is in the population you're sampling.

GILBERT WELCH: It is kind of a strange thing.

HARRIS: Dr. Gilbert Welch is a scientist at Brigham and Women's Hospital in Boston.

WELCH: An antibody test is much more likely to be wrong in a population with very little COVID exposure.

INSKEEP: Richard, I think we need to slow down here. Why would the...

HARRIS: OK.

INSKEEP: ...Accuracy of a test depend on how common the disease is in a population?

HARRIS: Yeah, that is surprising. But here's a simple way to look at it. Say you are running a test that gives five falsely positive results in 100 people. Sounds like pretty good odds, right? But...

INSKEEP: Yeah.

HARRIS: But consider this, Steve - if 5% of a population is infected and you run the test on 100 people, you should get five true positives, but you also have those five false positives.

INSKEEP: Oh.

HARRIS: And Welch says there's no way to know which is which.

WELCH: The tests will be wrong half the time. Half the people will be falsely reassured.

HARRIS: So it's basically a coin flip.

And it gets worse. The Food and Drug Administration does not regulate these tests, but the White House Coronavirus Task Force set an informal standard. They are supposed to have no more than 10 false positives per hundred. If you were to use a test that meets that standard in a population where only 1% of the population had been infected with the coronavirus, a positive result would be wrong a shocking amount - more than 9 times out of 10. There's a calculator on npr.org so you can run your own numbers.

And you can see that one way to limit this problem is to focus on populations where the disease is more common. Dr. Jordan Laser, a pathologist at Northwell Health on Long Island, N.Y., says it would make sense to start with health care workers.

JORDAN LASER: So it'd be wonderful for health care workers to know their immune status and give them just a peace of mind.

HARRIS: Positive results in a heavily exposed population are more likely to be true positives. Even so, Laser says it would still be a mistake to rely on these results.

LASER: Definitely don't use these tests to change your practices in terms of personal protective equipment. It really would be more of a psychological benefit.

HARRIS: Jeremy Gabrysch, the doctor in Austin, is also using these tests to satisfy the curiosity of his patients, not to provide advice about whether it's safe for people to expose themselves to the coronavirus.

GABRYSCH: We don't recommend that people interpret the result that way, and we counsel against that. I mean, it's not a Get Out of Jail Free card.

INSKEEP: Well, if it's not a Get Out of Jail Free card, is this test really useful at all?

HARRIS: Well, clearly it falls short of being the imagined passport that we would love to have. But you know, these tests can still be incredibly useful as long as individual false positive results don't matter. And one situation where that is the case is surveying broad populations. And in fact, these tests will be used to figure out just where the coronavirus is in the United States and how common it is.

INSKEEP: Richard, thanks so much.

HARRIS: Sure.

INSKEEP: NPR science correspondent Richard Harris. Transcript provided by NPR, Copyright NPR.