The TPR is uninformative in an epidemic dominated by asymptomatic transmission. And it doesn’t give us a clue what to do in Philadelphia during the hard winter ahead.
During the ongoing COVID-19 epidemic, public health practitioners and policymakers have increasingly relied on the test positivity rate (TPR) to decide whether to impose or relax constraints on social mobility and how much to expand testing capacity. Yet there is a genuine question whether the TPR – which basically gauges the number of positive cases as a percent of all persons tested – may be steering us off course.
During an epidemic where testing for infection is non-random and voluntary, the TPR may indeed tell us how well our testing program is identifying more severe, symptomatic cases. But in the current COVID-19 epidemic where asymptomatic persons are responsible for at least 40-45 percent of cases and play a dominant role in disease transmission, the TPR may not tell us how well we’re identifying infectious individuals.
When Testing Is Mandatory and Random
When testing is mandatory and random – for example, when the University of Wisconsin-Madison required all on-campus students to be tested regularly during an outbreak – the TPR may indeed accurately gauge the infection rate. If, say, one percent of all persons randomly tested during the past week come out positive, then we can reasonably estimate the current incidence rate per week to be about one percent. To be sure, there can still be false positives and false negatives, and there can be retesting of the same individuals. And we certainly don’t want to mix tests for active infection with antibody tests. While these concerns are valid, they are not fundamental.
When Testing is Voluntary and Non-Random
When testing is voluntary and non-random, however, it is widely acknowledged that the sickest, most symptomatic individuals will queue up for testing first. This behavioral observation has led to the conclusion that as testing capacity is expanded, the TPR will decline. If only the individuals in the first row in Figure 1 are tested, the TPR will be 40 percent (that is, 4 out of 10). If we expanded out capacity to test both rows, the TPR will be down to 30 percent (that is, 6 out of 20). This straightforward syllogism seems to imply that a low TPR is a favorable sign that our testing program “is casting a wide enough net.” Most authorities, including the World Health Organization (WHO), put the cutoff for a sufficiently low TPR at 5 percent, but some analysts peg the cutoff at 3 percent. Whatever cutoff is designated, the TPR is to be interpreted as “a measure of whether we’re doing enough testing.”
The Catch: A Large (and Growing) Pool of Asymptomatic Infected Persons
But there’s a catch. In the current COVID-19 epidemic, a lower TPR means our testing program is adequately identifying the sick people who are voluntarily queuing up to be tested. A further expansion of voluntary testing capacity may indeed lower the TPR, but it will not necessarily identify asymptomatic infectious individuals because they’re not even getting in line.
In Figure 2, we’ve included another group of 20 such asymptomatic people. Eight of them are positive and they don’t know it. If we really could test everyone, the TPR would be back up to 35 percent (that is, 14 out of 40).
In more technical terms, the critical assumption of a declining marginal yield to expanded testing breaks down when there is a significant pool of asymptomatic infected individuals with an extremely low propensity to get tested voluntarily. And as we’ll see shortly, the continued growth of this pool of asymptomatic infected individuals is the principal challenge underlying the hard winter months ahead.
Some Real Data from Philadelphia
Some real data from the city of Philadelphia in the United States, recently downloaded from its online dashboard, will clarify the point. At issue here is not whether Philadelphia’s current caseload is representative of other cities, but whether the problems in interpreting Philadelphia’s TPR are typical.
Figure 3 shows the weekly incidence of test-positive COVID-19 cases per 100,000 population in the city. To facilitate the presentation, we have partitioned the graph into four phases. (We’ve similarly partitioned the COVID-19 epidemic in Los Angeles County into four phases.)
In Phase I, from the week starting March 9 through the week starting March 30, 2020, highlighted in burgundy red, cases were rising rapidly from 3 to 178 per 100,000, equivalent to an average doubling time of 4.7 days. In Phase II, from the week starting April 6 through the week starting May 25, highlighted in green, the case incidence rate fell as emergency social distancing measures enacted in mid-March took effect.
During Phase III, from the week starting June 1 through the week starting September 14, highlighted in orange, the case incidence rate remained stable at about 50 per 100,000. Since then, with the arrival of Phase IV, highlighted in navy blue, the case incidence rate has topped 400 per 100, rising with an average doubling time of about 18 days. While reporting delays temporarily render the data for the last week in November unreliable, the case incidence has surely continued to rise.
The One-Case-per-Thousand-per-Week Threshold
Figure 3 includes an additional, horizontal dashed line at the level of 100 per 100,000 cases per week, equivalent to 1 case per 1,000 per week. This line crosses the epidemic curve during the week of March 23 (Phase I), the weeks of May 11 and May 18 (Phase II), and the week of October 12 (Phase IV). At each crossing point, the case incidence rate was the same, but the underlying dynamics of the city’s epidemic were different. Crossing the threshold from above (Phase II) is not the same as crossing from below, and crossing the threshold during the early surge of cases (Phase I) is not the same as a re-crossing when previously relaxed social distancing measures have proved insufficient (Phase IV).
Dynamic Pueyo Plot
Figure 4 plots the case incidence rate against the TPR, once again calculated on a weekly basis. The data points have been color-coded to correspond to the four phases identified in Figure 3. With the exception of a few points tightly clustered together in Phase III, the corresponding week is noted beside each point. (For visual clarity, the data point for the week of March 9 has been omitted.) The area of each point (not the diameter) is proportional to the total number of tests performed during each week, ranging from 2,147 tests during the week of March 16 to 60,980 tests during the week of November 16.
We’re calling Figure 4 a dynamic Pueyo plot because Tomas Pueyo appears to be the first analyst to plot case incidence against TPR. Pueyo’s version of the plot, however, was a static, cross-sectional comparison of different countries at a particular point in time, and not a dynamic rendering of the course of the epidemic in a single locality.
The Critical Question
The critical question is: What, precisely, does the additional information on the TPR in Figure 4 add to our knowledge of the dynamics of the disease propagation, the adequacy of testing, or the most appropriate epidemic-control measures in Philadelphia during the hard winter ahead?
Figure 4 tells us that the first time the epidemic curve crossed the 100-cases-per-100,000 threshold during the week of March 23, the TPR was 28 percent, far above the WHO’s acceptable level of 5 percent. The second time, during the weeks of May 11 and May 18, the TPR was 12 percent, still beyond the acceptable range. The implication is that the observed decline in case incidence during Phase II in the weeks after the mid-March declaration of emergency may have been an artifact of inadequate testing. Such a conclusion would defy the now-enormous literature on the effectiveness of the social distancing measures put into effect during Phase II.
The fact that the orange data points in Figure 4 are situated at or near the 5-percent TPR threshold would seem to imply that Philadelphia had adequate testing during Phase III. But if that conclusion has any substantive meaning at all, then it must apply as well to the first four weeks of Phase IV, when the number of cases per week more than doubled. Does this mean that no additional aggressive testing measures were required?
It is now evident that during the week of September 21, Philadelphia was on the verge of a new and more deadly resurgence of infections, yet its TPR of 2.5 percent was telling us everything was just fine.
Counterarguments
The raw number of positives indeed obscures the number of negatives or the number tested. And there isn’t the remotest doubt that current counts of positive test results substantially understate the actual incidence of infection. Still, when it comes to assessing the dynamic state of the epidemic in Philadelphia or anywhere else, the number tested may be a more misleading denominator than the standard population count.
One might argue that the Philadelphia data are entirely consistent with the TPR paradigm. The expansion in testing seen in the last two months – as indicated by the enlarging blue data points in Figure 4 – was accompanied by an increase in the TPR. This observation, some analysts may contend, precisely illustrates the value of the TPR in identifying a dangerous outbreak. The problem with this logic is that both the number of positive tests and the number of total tests are endogenous variables. Without a natural experiment, we can’t tell whether new positive cases were pushing total testing, or whether expanded testing was pulling the number of case counts. Another analyst might observe, “The percent positive will be high if the number of positive tests is too high, or if the number of total tests is too low.” But that’s just having it both ways.
The Hard Winter Ahead
The first equation of the classic SIR model of epidemics teaches us that the incidence of new cases is a product of two basic factors: (1) the number of contagious individuals and (2) the rate at which these contagious individuals transmit their infections to other susceptible individuals.
During Phase I of the epidemic in Philadelphia, Los Angeles, New York City, San Antonio, Broward County, Florida, and countless other metropolitan areas throughout the world, the surge in cases was driven by factor #2. At the very start, after all, we had no social distancing measures, so that a relatively small number of contagious persons could propagate their infections widely.
During this Phase IV of the epidemic – which appears to be continuing into the cold, hard winter to come – the new surge in cases is being driven by factor #1. The growing number of asymptomatic individuals has now overwhelmed what social distancing measures we have put in place. Despite marginal improvements in testing capacity of the order of magnitude seen in Philadelphia, we haven’t come anywhere near the level of testing we need.
Without routine, repeated testing of asymptomatic carriers through widespread rapid testing on a far more massive scale than we’ve seen so far, we’ll continue to be “operating in the dark,” no matter what the test positivity rate is.
Comments and Responses
In response to the thoughtful comment below by Prof. Raphael Thomadsen. As an indicator of the adequacy of testing, the TPR appears to have little or no informative value. When it comes to gauging the severity of the epidemic, the TPR might serve as a proxy for the case incidence rate. But that just begs the question: What does the graph of TPR in Figure 4 tell us that we haven’t already learned from the graph of the case incidence rate in Figure 3? As to whether there were different test regimes, take a look at the graph below. There was clearly a different regime during Phase I, when the CDC was still struggling to come up with a reliable PCR test for coronavirus infection. After that, the total number of tests has climbed steadily at about 5 percent per week.
Tweets
Life Under Quarantine 
I think the headline here is too harsh. I had assumed that the positivity rate was useless for the reasons you lay out. However, looking at your graph, it seems more that there are different test regimes, and that on a day-to-day level the positivity rate is a really good indicator about how COVID is changing day-by-day. It seems then, that it depends on what your goal is, but for short-run analysis it looks like the positivity rate may be a fantastic predictor.
LikeLike