As Maryland officials continue to combat the coronavirus pandemic, they have continually pointed to a shrinking testing positivity rate as a sign of the state’s improvement, with Gov. Larry Hogan touting the metric on his Twitter account almostdaily.
But the figure the state routinely points to has recently been far beneath that of third-party sites tracking the pandemic nationally, including Johns Hopkins University’s coronavirus resource center.
For more than 75 straight days, Maryland health officials have reported that the state’s seven-day average testing positivity rate for the coronavirus has been beneath 5%. But others have Maryland shifting above and below that mark over what is nearly a three-month period.
The 5% benchmark is significant, given that the World Health Organization recommends governments have a rate beneath that point for 14 straight days before easing virus-related restrictions.
Yet while the state has reported being beneath that threshold every day since June 26, Hopkins’ coronavirus resource center showed Maryland above 5% from July 10 to Aug. 14, then again beginning Sept. 5.
Through Tuesday’s data, when the state’s figure was below the threshold at 3.68%, Hopkins had Maryland at 6.1%, above 6% for a second straight day. The gap between the two figures has not been wider since late May, when both rates exceeded 13%.
Hopkins, which also reports the rates of the other 49 states, Puerto Rico and Washington D.C. using data from The COVID Tracking Project, is determining the rate with a different methodology than Maryland.
During an online meeting of a General Assembly coronavirus work group Aug. 12, some Maryland lawmakers asked Dr. Jinlene Chan, the state’s acting deputy health secretary for public health, about the differences in positivity rate reporting. Chan said Maryland has used the same methodology from the start of the pandemic to stay consistent.
“We picked a methodology that we believe is the most correct and most stable for us,” Chan told lawmakers.
She acknowledged, however, “there are multiple data definitions and multiple ways in which states are handling their own data.” As a result, “it makes it very difficult to compare apples to apples.”
Here’s what you need to know about how the rates are calculated, and whether the difference matters.
Why are the rates different?
It all boils down to the fact that Maryland and Hopkins are calculating a percentage from two different data sets: total tests administered vs. people tested.
Maryland determines its rate based on the total number of tests administered. During the nearly six months of the pandemic, the state has reported performing more than 2.1 million tests as of Wednesday. About 700,000 of those tests — whether positive or negative — have been performed on people who had already been tested at least once. Those retaken tests, as long as they were not taken by the same person on the same day at the same location, are included in the state’s math. Maryland does not include antibody tests, which measure if someone previously had the virus.
The state calculates its seven-day rate by taking the number of new positive tests — which is typically higherthan the number of new cases — each day, dividing the number of total tests performed that day, and averaging those figures over a week. Maryland’s coronavirus dashboard does not include the exact number of daily positive and negative tests, but does say what percentage of that day’s total test results were positive.
The U.S. Centers for Disease Control and Prevention guidelines define positivity rate, or percent positivity, as “the number of positive tests divided by the total test results, with total test results defined as the sum of positive tests and negative tests.” However, the CDC does not specify which tests should be included as “the number of positive tests” or “total test results.”
The CDC’s recommendations for reopening reference both positivity rate and median test result return time, a statistic that Maryland, like other states, is not making public.
Hopkins determines the state’s positivity rate using people, not tests, meaning those 700,000 repeat tests aren’t included in its calculations. Hopkins divides the number of cases by number of people tested, or the sum of people who have newly tested positive and newly tested negative.
Dr. Jennifer Nuzzo, lead epidemiologist for Hopkins' coronavirus resource center’s Testing Insights Initiative, said Thursday that the growing difference between Hopkins' figure and the state’s directly relates to the inclusion, or lack thereof, of those repeat tests.
“My hunch is that what’s probably happening now is that there’s just more repeat testing that is occurring and being captured in the state’s calculation that’s not being captured in ours. ... If our number is going up versus theirs, what’s probably happening is that the number of people that are newly being tested that are testing positive is increasing," Nuzzo said. "These are people who are possibly being tested for the first time, or they’ve had some kind of status change, meaning they tested negative before but now are testing positive.”
According to state data, only 1.3% of the repeat test results reported in the past week have come back positive, helping decrease the state’s overall seven-day rate relative to Hopkins'.
”Someone who was hospitalized, possibly not even for COVID, may have 10 tests in the course of that hospital stay,” Nuzzo said last month. “We’re less interested in tests two through 10.”
So why do Maryland and Hopkins use different data sets?
Hopkins’ coronavirus resource center isn’t focused on only Maryland. It compiles data from every state in the U.S. to show which are meeting WHO’s recommended testing criteria.
The problem is, not every state releases its coronavirus metrics in the same way. Some don’t provide exactly how many people tested negative, instead only offering the number of cumulative negative tests. The desire to use a similar approach from state to state keeps Hopkins from matching individual state’s formulas, including Maryland’s.
“We can’t use the state’s method because not even half the states report the data that would be required to use the state’s method,” Nuzzo said. “Every single state does it differently, and there are no federal data standards, so what we think needs to happen is the federal government or some national entity needs to tell states, ‘Here is how positivity should be calculated, here are the categories of data that should go into that calculation, and here is how you should count either people or tests that go into the category that you’re reporting,’ so that we don’t have the situation where multiple states are using the same terms but the data that go into those terms are completely different.
“Now, if every state reported the data similarly, that might warrant a different positivity calculation just for the sake of consistency, but we are very much encouraging of national leadership to standardize both the calculations and the data that go into them.”
Because of the inconsistencies, Hopkins says its rate is determined using the “number of cases divided by number of negative tests plus number of cases.” In Maryland’s case, Hopkins uses people who test negative in place of negative tests.
Early in the pandemic, Maryland referred to its negativity figure as the number of negative tests, but since May, its dashboard has featured the same data as “persons tested negative.”
On its coronavirus resource center webpage, Hopkins states: “We feel that the ideal way to calculate positivity would be number of people who test positive divided by number of people who are tested.”
But Nuzzo made sure to point out that the state’s method is an acceptable one.
“To answer the question of if the state is hiding things or cooking the books, I mean, they’re using a method that many other states do,” she said. “It’s a completely valid approach.”
Positivity rate can tell the state whether it’s testing enough people and whether it’s testing the right people, Nuzzo said. But although both the WHO and CDC include positivity rate in their metrics for determining reopening processes, it should not be the only statistic local governments use, Nuzzo said, also citing cases, hospitalizations and others.
“We look at positivity to give us a sense of how well we’re doing on testing, whether we’re casting a wide enough net to find the infections that we think are out there,” she said. “Positivity not only tells us whether we’re doing enough testing or maybe we need to do more, but in seeing what our positivity is and whether we’re casting a wide enough net, it can also help better interpret our case numbers.
”We have to recognize that no metric is perfect. There’s not going to be one single number that tells us how we are. Unfortunately, public health surveillance requires looking at multiple data inputs and kind of triangulating our way to the truth.”
Breaking News Alerts Newsletter
As it happens
Get updates on the coronavirus pandemic and other news as it happens with our free breaking news email alerts.
Despite the differences between them, it’s not insignificant that both rates have dropped dramatically over recent months, though both have trickled upward in recent days. Using either calculation method, Maryland’s positivity exceeded 20% four months ago. In early June, both rates were above 8%.
Even as the figures remain more than 2% apart, Nuzzo said Thursday Sept. 10, the difference is relatively small when placed into the scope of the large rates some states saw earlier in the pandemic and are still seeing now. Through Wednesday Sept. 9 data, Hopkins had 12 states with seven-day rates in double-digits.
“I don’t think they’re hugely apart," Nuzzo said. "I think part of why they appear to be so far apart is ours has the state above 5%, and theirs is below. In terms of the range of positivities we’ve seen throughout the course of this epidemic — back in the spring, we saw double-digit, 50% in some states — it’s actually quite small in the grand scheme of things. I don’t think it’s necessarily a cause for concern. It’s something we want to understand and track and try to figure out why, and then depending on the answers to those questions, we may want to do more targeted testing in different places.”
The gap between the two has widened in recent weeks as Maryland has started to report more test results daily, but both rates are much better than they once were.
“In my view, what is more important than the exact number is the trend in the numbers,” Nuzzo said. “Even if the state is at 1%, if their positivity starts picking up over time, that to me would be worrying on its own because we would want to understand why that’s happening and use that signal as an early warning that we potentially need to do something before it gets much larger. The directionality of positivity is perhaps more important than the exact number.”