How lethal is COVID-19 compared with the common flu? To answer this, health officials first have to know how many people in a community are infected, or have been in the past. They’re using virus swab tests to identify people who have COVID-19, and they’re administering antibody blood tests to detect who has had the infection previously.
The accuracy of antibody tests is still in question, however. Inaccurate test results could produce inaccurate infection counts, which would then produce inaccurate prevalence and mortality rates. But Chicago Booth’s Panos Toulis suggests a statistical method that officials can use to arrive at more accurate counts and thereby craft more-nuanced containment policies.
Antibody testing isn’t yet widespread, so officials are trying to extrapolate from studies conducted in certain locations, including Santa Clara County, California, and New York State. In Santa Clara, officials tested 3,330 people, of whom 50 tested positive for COVID-19 antibodies. After some reweighting to make the sample more representative of the general population, a research team led by Stanford’s Eran Bendavid estimates that 2–4 percent of the general population in the area was likely infected.
But how robust and trustworthy are these results? The Santa Clara study used a standard statistical method that Toulis compares to a Swiss Army knife. Standard methods, which involve assumptions, can serve many purposes and are great if you need one tool to do many things, he says. When possible, you reach for a tool more specific to the task at hand.
Toulis reexamined the results using a different method, aiming to eliminate unnecessary assumptions by including two unknown factors as parameters. One was the false-positive rate—if someone does not have COVID-19 antibodies, what is the probability that the test result will still be positive? The second was the true-positive rate—if someone has the antibodies, what is the probability that a test will come out positive? “Our target is disease prevalence. In between, there are these two numbers we don’t care as much about but that help solve the equation,” he explains.
This gave him three unknowns, which functioned as parameters for his problem: the true-positive rate, the false-positive rate, and the prevalence rate. He then looked at all possible values of the unknown parameters and assessed the likelihood of observed test results assuming any given values of the parameters. For example, what is the likelihood of seeing 50 positives out of 3,330 trials when the true positive rate is 85 percent, the false-positive rate is 0.8 percent, and the prevalence rate is 5 percent? He had some sense of the actual true- and false-positive rates because Santa Clara researchers, using blood collected before the COVID-19 crisis, conducted a validation study that gave them a better sense of those rates in their sample.
