The coronavirus pandemic is nowhere near to being over, yet states are starting to ease restrictions to try to avert economic disaster.
Public health experts say the only safe way to reopen the country for business as usual is to follow the gating criteria set up by the White House Coronavirus Task Force, and to test, test, test.
There are two distinct types of coronavirus tests you’ve probably heard about. One is the molecular diagnostic test that can test for the virus itself, commonly called an rt-PCR test. The other is a serologic test – a blood test – that detects antibodies, which are part of your body’s reaction to the virus.
While both tests are key, it’s the virus test, those rt-PCR tests or molecular diagnostic tests, that are part of the gating criteria to reopen the country. They are the tests that determine if someone currently has the virus in their body, if they need to be isolated and whether their contacts needs to be traced and potentially quarantined. Not all contacts of someone who tests positive will need to be followed up on and traced to limit the spread of the virus – rather, it’s influenced by how long the infected person was in contact with the other person, and in what environment. Some of these contacts will also have to be tested for the virus. Figuring all that out is a laborious task, and will potentially require hundreds of thousands of people to serve as contact tracers all over the country.
Amid our need to get more tests out, however, there are some fundamental problems with these diagnostic tests that may be overlooked.
One is the speed of the testing, which may take up to several days to get results. This is a potentially catastrophic shortcoming. If someone is harboring the virus and has to wait several days for a result, that person could possibly come into real contact with dozens of people, unless they proactively isolate themselves, which many may not do, especially if they are feeling fine or asymptomatic.
But I wanted to focus on another stunning problem many Americans probably haven’t even heard about, and that’s the accuracy of the tests. Or, rather, the lack of accuracy.
“We have many tests on the market that we have no idea how well they work,” Mike Osterholm, director of the Center for Infectious Disease Research and Policy at the University of Minnesota, told us.
In part, the problem started when the U.S. Food and Drug Administration gave emergency use authorization to 56 tests for coronavirus – 46 of those for molecular diagnostic tests. While there was an understandable rush to increase testing, it also meant the FDA had not independently evaluated and validated them to see if they really work properly.
And, as we are learning, some of them do not.
“And that’s the challenge in terms of understanding, if you get a negative result, is it really negative?” Osterholm asked.
To understand the problem, it’s important to understand a technical term: sensitivity.
Sensitivity is the ability of a test to correctly identify those currently infected with the virus. If a test has a high “false negative” rate, that means it has poor sensitivity. It is not “sensitive” enough to pick up all the patients who are infected with the virus.
To better understand why that would happen, it is helpful to understand how these tests work. The diagnostic rt-PCR test is the one that requires the swab, or in some cases saliva. (The antibody test will require blood, either from a blood spot or from a vein. That’s a good way to tell them apart). The “rt” means reverse transcriptase. Because the novel coronavirus is just a little strand of RNA, the first thing that needs to happen is to transcribe that RNA back into DNA, which is done through an enzyme in the testing kit. After that the polymerase chain reaction “amplifies” the DNA so that it can be more easily detected.
This is the gold standard of testing, but as Osterholm and Mark Olshaker wrote in the New York Times, the way those tests are performed may be inadequate. For example, “the FDA recommends 40 cycles of amplification, but even after those, too little of the virus’s genetic material might be present to be detectable,” they write.
There are also several other factors that influence the accuracy of the tests, including the time from onset of illness, the concentration of virus in the specimen, the quality of the specimen collected from a person and how it is processed, and the precise formulation of the reagents in the test kits.
Dr. Gary Procop, chief of virology at the Cleveland Clinic, evaluated five of the tests used to diagnose Covid-19 patient after he and his staff noticed they were getting varied results depending on what tests they used. He led a team that evaluated over 200 samples using the five tests. And what did they find? Test sensitivities ranged from 85% to 100%
Procop told CNN that the US Centers for Disease Control and Prevention test was 100% accurate. He also pointed out tests from Cepheid and Roche both tested with sensitivity over 95%.
However, the highly touted Abbott ID Now – a rapid diagnostic that is essentially a lab in a box – scored much lower. While it can produce results in minutes, it only scored with a sensitivity rate of 85%. That’s a 15% false negative rate. Another analysis found that it potentially had a 25% false negative rate – meaning it could miss up to 1 in 4 positive cases.
On Thursday, Dr. Francis Collins, the director of the National Institutes of Health, testified before the Senate Health Education Labor and Pensions Committee about the implications of the false negatives rates.
“If you’re in a circumstance where you really, really don’t want to miss a diagnosis of somebody who’s already carrying the virus, you’d like to have something that has a higher sensitivity than that,” Collins said.
In a hospital, a false negative rate of 15% to 25% could mean putting an infectious person into a regular hospital ward. It could also lead to an infected patient being sent back to a nursing home full of vulnerable people or even into the public in the dozens of states that are prematurely reopening. “These are the people that we are concerned about,” said Osterholm, “going away from that testing procedure thinking ‘I’m fine, I don’t have to worry about transmitting,’ when in fact, they do.”
When we asked the FDA if getting tests out quickly impacted the quality of the tests, an agency spokesperson told us the FDA’s oversight doesn’t end with an EUA and that it continues to track these tests and take action whenever necessary.
Also keep in mind, that regardless of the test’s sensitivity, you may still test negative one day, and test positive a following day. The diagnostic test is a snapshot, and can only let you know you are negative at that specific time.
The study by Procop and his team hasn’t been peer-reviewed yet, but they are expecting to publish their findings soon. He also noted that since he and his team tested the diagnostics, Abbott came out with a warning that the sensitivity of its test is impacted by the chemical used to hold the virus sample – known as viral transport media.
In a statement, Abbott said “This method can reduce the sensitivity of the test through dilution, which can potentially lead to false negative results.”
The company added “ID NOW is designed for and typically used within urgent care clinics, physicians’ office labs, emergency departments and retail settings. This means that patient samples (swabs) are collected and tested directly on the ID NOW instrument, near the patient and without the use of VTM. (Viral Transport Medium).”
Abbott also pointed to a recent report from the Detroit Health Department, which found that results from the Abbott ID Now matched with those of a traditional PCR test 98% of the time.
So, where do we go from here?
“They need to review these tests carefully and basically take the ones off the mark that are not working well,” Osterholm says.
It’s an idea that Procop also suggested. The FDA could “start assessing these data and ask manufacturers to do it, so that the performance characteristics are really transparent to everyone going to use these.”
Right now, while manufacturers do have to share sensitivity data with the FDA, but just how much they have to share with the public is unclear. These are companies self-reporting their accuracy rates – how do they determine them? How do they compare to each other? What is the standard? This is now vital information for everyone. We shouldn’t have to rely on strapped teams, like Procop’s at Cleveland Clinic, to study this. We need the FDA to step in. Clinicians are asking for it.
“It is now time to start a more traditional FDA process where we start looking at a test characteristics and define those against what we would consider a gold standard,” said Procop.
As Osterholm told me, “The bottom line is that this is a major requirement that the FDA has to meet. They cannot abdicate their responsibility here. We need clear and compelling information on how well these tests work. And if they don’t, then laboratories need to be aware of that and not use those tests.”
In our clamor for more tests, we need to remember that a bad test is potentially worse than no test. And we need the data to know which tests are better than others.
The good tests must be supported, made widely available at locations throughout communities – pharmacies, kiosks, public settings, and maybe even in people’s homes. They must be performed regularly, and of course, they must be highly accurate.
On Friday, the FDA granted the first emergency use authorization to an antigen diagnostic test for the coronavirus. This test can quickly detect fragments of the virus proteins by testing samples collected from a nasal swab. Unlike rt-PCRs, which can take time and materials, antigen tests can turn around results much more quickly. However, antigen tests aren’t as sensitive as rt-PCR diagnostics and they can yield more false negatives.
And that brings us back to the original problem, the persistent shortage of tests in the United States. So far, the United States has performed over 8 million tests, according to the Covid Tracking Project. If one person took just one test – that would translate to about 2.5%. But, one person can be tested multiple times, so we really don’t know what percent of the population has been tested.
Get CNN Health's weekly newsletter
Sign up here to get The Results Are In with Dr. Sanjay Gupta every Tuesday from the CNN Health team.
Harvard’s Safra Center for Ethics, along with a group of bipartisan public health and economics experts, released a roadmap that calls for 20 million tests a day by mid-July to fully remobilize the economy. That is roughly 6% of America being tested, every day.
Another way to think about it: That’s like testing the entire country every 14 days.
Bottom line: We need lots of testing. Those tests have to performed regularly. The results have to come back quickly. And the accuracy has to be stellar.
It is audacious, but without a highly effective medication to treat Covid-19 or a vaccine to prevent the virus, this quality of testing is the most reliable way to restore some sense of normalcy. It will not only be helpful at containing the virus, but also give people the psychological confidence that they aren’t spreading the virus unintentionally. This is how we achieve a reasonable baseline of safety to start reopening the country.
CNN’s Nadia Kounang and Maggie Fox contributed to this report.
