A week-long rally attended by half a million motorcycle enthusiasts last month in Sturgis, South Dakota could have eventually led to an extra 250,000 cases of covid-19, according to new research released by a team of economists. But though it’s certainly likely that the rally was a superspreading event, this particular estimate may be off the mark, as two scientists not affiliated with the study explained to us.
The Sturgis Motorcycle Rally is an annual event held since 1938. This year, despite concerns of the large crowds being a conduit for covid-19, the rally ran from August 6 to 19 and is thought to have attracted around 460,000 people from all over the U.S. Infamously, Steve Harwell, the lead singer of Smash Mouth, mocked the seriousness of covid-19 while performing to a packed crowd during the rally.
By early September, at least 260 covid-19 cases were reported to have been linked to people attending the rally, along with one reported death. Most of these cases involved people in South Dakota, though some were traced back to people in 11 other states; the one death was a man in his 60s from Minnesota. But, at least according to this new paper, the true covid-19 spread from the rally is much higher than reported.
The researchers analyzed several different sources of information to come up with their estimates. This included anonymous cell phone location data of locals and travelers who attended the rally, foot traffic data to bars and restaurants, and data on reported covid-19 cases collected by the Centers for Disease Control and Prevention.
The team found that the Sturgis rally was associated with lots of visitors from out of town, more foot traffic to bars and restaurants in the area, and increased movement outside of the home by local residents. These factors combined to promote the spread of the contagious illness during and after the rally, they found, which led to a noticeable jump in cases in South Dakota and in areas where larger numbers of attendees came from.
All told, they calculated that Meade County, home of Sturgis, had an extra six to seven cases of covid-19 per every 1,000 people than it would have normally by September 2. Counties with the highest number of residents who went to the rally were also said to have experienced a 7% to 12.5% increase in covid-19 cases, relative to similar counties where no people traveled to the rally. These spikes ultimately amounted to an added 250,000 cases, they concluded, amounting to nearly 20 percent of all cases believed to have occurred in the U.S. between August 2 and September 2.
The paper was released recently as a preprint by the IZA Institute of Labor Economics, a private research group focused on labor markets that’s based in Germany. Its authors include several economists from San Diego State University’s Center for Health Economics & Policy Studies, the University of Colorado Denver, and the National Bureau of Economic Research. Preprints are preliminary research, meaning that they haven’t gone through the typical peer review process.
While right-wing folks on social media were quick to dismiss the results as politically motivated, this same group of economists also found little evidence for a spike in cases linked to the Tulsa, Oklahoma rally for President Trump’s reelection held in late June (some experts have argued otherwise, however). They similarly found little data to suggest that the widespread Black Lives Matter rallies held earlier this summer were linked to a substantial increase in cases in the cities where they held.
According to Elizabeth Stuart, a statistician and associate dean at the Johns Hopkins Bloomberg School of Public Health, the basic methodology used by the team is commonly deployed to evaluate the effects of policy interventions. But there’s a lot of uncertainty in applying these tools to studying how a never-before-seen disease like covid-19 might spread through a population—uncertainty that she felt the researchers could have done a better job of showing.
“I think they did a reasonable analysis. But there were many choices made along the way that could have been made differently,” said Stuart, who is not affiliated with the research.
One example she brought up was how the team compared South Dakota to other states like Vermont and New Hampshire in trying to create a “synthetic control,” a statistical tool that tries its best to imagine what would have happened in the same place had variable A (the rally happening in Sturgis in this case) never occurred. Because the states used were relatively small and may be different from South Dakota in many important ways, Stuart said, the exact numbers produced by the team’s model might not be as stable as they appear, even if the basic finding that the rally created a superspreading event is true.
“So the big, high-level conclusion from the study is that there were over 200,000 cases from this and that it accounted for 20% percent of cases nationwide. But there’s no uncertainty given to that—there’s no confidence interval,” Stuart said, referring to a common statistical method used to present the likely range of possibilities for any one result. “And I really think it’s important in these contexts to convey the uncertainty, both the statistical uncertainty and also the more fundamental uncertainty of their model.”
Joshua Salomon, a health policy researcher and professor of medicine at Stanford University, told Gizmodo via email that the team’s analysis is “thoughtfully designed” and that he agrees with the general conclusion that the rally likely led to a large number of infections. But he said comparing counties that did or didn’t have people who went to Sturgis as a way to estimate the number of cases caused by the rally, as the authors also did, has its shortcomings too.
“A key limitation is that these sets of counties appear to have been on different trajectories before the rally, which makes it much harder to compare the trends after the rally and attribute the difference to the rally,” he said. “So, the 250,000 estimate is a nice headline-grabber but pretty likely to be an overestimate.”
The authors do acknowledge that their numbers may not be a precise accounting of the covid-19 fallout that might have directly come from the rally. It’s possible, for instance, that some people who caught covid-19 during or after the rally would have caught it anyhow in some other way.
“However, this calculation is nonetheless useful as it provides a ballpark estimate as to how large of an externality a single superspreading event can impose, and a sense of how valuable restrictions on mass gatherings can be in this context,” the authors wrote.
The Sturgis gathering had some key differences from other large gatherings that have had people concerned about superspreading. For one, the Sturgis rally lasted far longer than any single protest or political rally and was attended by far greater numbers, with people coming largely from outside of the state. Anecdotally, few people who went to Sturgis were said to have worn masks or practiced much social distancing while there. And perhaps, unlike the BLM rallies, local residents also started relaxing their distancing and staying at home less during and after the rally, as the authors’ earlier research had suggested.
These factors seem to have turned the Sturgi rally into a “situation where many of the ‘worst case scenarios’ for superspreading occurred simultaneously,” the authors wrote. Alternatively, they provide hope that regulations like mandating mask-wearing, limiting the size of gatherings, and restricting indoor dining and drinking in bars can prevent these superspreading events from happening in the first place.
Stuart, for her part, doesn’t disagree there.
“Basically, I don’t have full confidence in how precise their estimate is. But I think it’s highly plausible that there was this spread. And certainly the recommendations around mask wearing, being outdoors, and still letting people gather but doing it in ways where it is safer—I think it’s the right kind of message,” she said.
Correction: A previous version of this article incorrectly said “Tulsa, Arizona” instead of “Tulsa, Oklahoma.” Thanks to commenter Bill for pointing out the typo.