June 29, 2020, by SSSC

Testing is a critical part of our plans to cope with COVID-19 (technically, SARS-CoV-2) while conducting business during the pandemic. We need testing because we can’t see the virus and it is the only way we can be sure that someone is infected or has been infected. More importantly, testing is necessary because people are able to spread the disease even when they are not showing any symptoms. As of June 18^th the Center for Disease Control (CDC) reported about 25 million tests had been administered and 10% of those were positive. That is a lot of tests, but it is not clear what those data really mean.

In order to better understand how we might think about testing, it is important to understand some of the anticipated and unanticipated issues associated with the tests. First, there are different types of tests that tell us different things, but the CDC lumps them together when it reports the number of tests. Second, like all tests, they  are not perfect, so we don’t know how many of the negative tests might have been wrong. Third, in our haste to roll out testing we have encumbered a variety of issues associated with test quality and sampling design further limits our ability to interpret the results..

Different types of tests and what they tell you

There are two broad classes of tests (see our infographic): Molecular Tests that look for genetic material from the virus in swabs taken from your upper respiratory tract and Serological Tests that look for antibodies found in blood samples. Currently there are two varieties of Molecular Tests:  the “RT-PCR test”, is completed in an analytical laboratory so it takes a day or so to get the results back. “RT-PCR” stands for reverse transcriptase polymerase chain reaction. The “Point of Care” molecular tests are the second variety. They can be run on-site in a several minutes. A positive result from both these types of tests indicates that the virus is reproducing in your respiratory tract and means you are actively infected with COVID-19.

The Serological Tests examine your blood to see if you have developed antibodies to the virus. A positive test here means that you have been exposed to the virus and developed an immunological response. There are two types of Serological Tests, the first being the Rapid Diagnostic Tests (RDT) which give you a yes or no for antibody production.  The second relies on an analytical method called ELISA (enzyme linked immunosorbent assay) that quantifies the amount of antibodies present. These tests may also indicate if you have immunity to the virus, but scientists are still trying to understand if exposure to the virus means you will be immune to further infection.

Testing is not perfect

These tests, like all analytical tests, are imperfect. The testing processes are complicated and require exacting skills in each of the many steps involved. There are an infinite number of ways things can go awry. Looking at the table below there are four ways they can turn out- relative to the true condition of the patient. If the patient is infected and the test is positive then it is called a “true positive”, alternatively if the test is negative (and the patient is actually infected) then the test has produced an error called a “false negative”. Similarly, an error is encountered when the patient is not infected but the test says they are. This is called a “false positive”. Finally, if the test is negative for the uninfected patient the result is called a “true negative”.

For the molecular tests the percentage of false negatives is critical because these are people that would be told they are free of disease when they are, in fact, carrying it. This could happen if you had just been infected but weren’t far enough along to start spreading virus.  For example, Johns Hopkins University reported that if you were exposed within 4 days there was a 67% chance that the test would give a false negative. They found the fewest false negatives occurred about 3 days after symptoms developed or about 8 days after exposure and even then, the false negative rate was 20%. That suggests that out of 24 million or so negative tests reported for the US by the Centers for Disease Control on June 18th, somewhere around 5 million have been false negatives.

The critical issue for the serological tests is the percentage of false positives. If the test says you have antibodies when you really don’t then you may place yourself in conditions ripe for exposure and think you are protected. Unfortunately, these tests also detect the presence of antibodies to other corona viruses in addition to COVID-19. Consequently, antibody tests are prone to producing false positives and the Center for Disease Control advises that these tests should not be used to clear people of exposure.

The rates at which tests produce false positives or negatives are usually measured under controlled conditions. For example, testing blood samples known to be free of antibodies allows you to see how many times the test erroneously finds antibodies present and provides the false positive rate. Similarly, testing blood from people known to have recovered from COVID allows you to estimate the false negative rate. If you do this then you can calculate the accuracy of the test by adding together the number of true positives and true negatives and dividing by the overall number of tests, but this describes accuracy under clinical conditions. It does not represent the likelihood that a test is accurate in a random sample from an infected population because you have to also account for prevalence of the disease in the population you are testing. As it turns out, if the test is 95% accurate but the prevalence of the disease in a given population is much lower than 5% then the test is not very useful. A recent report from Canada calculated that molecular tests with more than 95% accuracy are reliable only when 9% or more of the population has the virus.

Test quality and sampling issues

These theoretical calculations call into question the quality of the tests being used and how they are being used. There are now a large number of tests on the market. Some of which have been approved by the FDA under Emergency Use Authorizations (EUAs) and others which have not. On one level this supports our need for increased testing, but it also creates a problem because we don’t always know the error rates for these tests. That makes using test data tricky if you want to account for false positives and false negatives. In early May, Vox reported there were 59 Molecular Tests approved by the FDA under EUAs and another 100 pending. These tests are made by different companies and are not interchangeable. They use different proprietary reagents (chemicals) and likely have very different statistical properties. We don’t know these properties because the EUA’s don’t require that they be as rigorously tested as usual. 

Sample design is another important issue. If the tests really only provide reliable results when some percentage of the population has the disease then what do we mean by “population”?  The molecular tests are offered primarily to patients thought to have been exposed or those exhibiting symptoms. So, the “population” being tested probably has an adequate prevalence and test results are likely reliable. While testing this way is valuable for identifying who has the disease it is of limited utility for understanding trends in infection rates. That requires a different approach to testing that includes random testing of populations.

Recently some governors have claimed that infection rates are increasing in their states only because we have increased the number of tests. It is certainly true that as the number of tests goes up there will be a greater number of infections detected. However, it is important to consider the proportion of tests that are positive, or the “positivity” of the tests. If the number of tests increases and the positivity stays constant or goes down then you can bet that infections are not increasing. On the other hand, if the number of tests goes up and the positivity also increases then the number of infections must be increasing.  This is what is happening in Florida right now. According to the Johns Hopkins COVID tracker the number of tests increased slightly between June 5 and June 18 while the number of daily detections and positivity has increased by a factor of three.

Testing is critical for making our way through the pandemic. Like all scientific data test results are easy to misinterpret or misuse. It is important to always consider what the test results mean to the individual and what they reveal about the population. It is important to understand the accuracy of the tests and the conditions that optimize their accuracy in order to fully grasp their reliability. I have tried to outline some of the issues in this essay, but there are many more. My goal is to demonstrate the complexity of the testing issue and that the details are important to consider.  Our ability to test has increased significantly over the last few months and our ability to conduct tests is invaluable in monitoring the progress of the pandemic, but as in all things associated with this pandemic, it is best to be careful and stay informed.    

What you can do

In Alaska the State is encouraging anyone with symptoms to contact their health provider to see if they should be tested and providers are being encouraged to test anyone with COVID19 symptoms. If you are in close contact with someone with a confirmed case, then the State is encouraging you to get tested. There are testing locations throughout the state. Be sure the check out the State’s Testing Guidelines at : http://dhss.alaska.gov/dph/Epi/id/Pages/COVID-19/testing.aspx#guidelines