Opinion: Real-world data, not predictions, should drive decisions on Covid-19 and school opening

The United States has exceeded 31 million Covid-19 infections (a messy data point) and is approaching 570,000 Covid-19 deaths (a more robust data point). Yet despite the abundance of data about the pandemic, the best available information is not usually what guides policymakers.

Editor’s note: Two of the authors, Leslie Bienen and Eric Happel, failed to disclose their affiliations with ED300, an organization that advocates for opening Oregon schools and sports. That has been updated in the bio at the end of the article.

Some policies are senselessly cruel, such as keeping family members from visiting loved ones dying of Covid-19. Others heighten disparities in income, health, and education.


After nine months of observing school closures and reopenings, we identified two factors that appear to be influencing decision-makers toward making less rational, less effective school-reopening policies: overreliance on alarming “predictive” models that are not actually predictive, and media reports based on data that are poorly analyzed and then manipulated to fit preconceived negative narratives.

We propose three simple solutions to address these factors.


Don’t use doomsday scenarios based on flawed models for planning purposes

Models that later turn out to be inaccurate have distorted Covid-19 policies since the pandemic started. Such models have been cited repeatedly by school administrators, legislators, and governors as reasons schools should close, or remain closed.

In January 2021, for example, a biostatistician modeler serving as a consultant to a large school district in Oregon gave a presentation to a widely attended public school board meeting. His model predicted a large local spike in Covid-19 hospitalizations in February and March, based on assumptions about variants and mitigation fatigue. This model showed hospital cases in Oregon nearly doubling between January and March.

In a video of the meeting, school board members can be seen asking fearful questions about this grim scenario. One board member comments, “My fear is we are going to get both the fatigue and the variant.”

Yet between January and March, hospitalizations in the region actually fell by 66%.

This school district maintained fully remote learning into April, despite some of the lowest Covid-19 rates in the country. Middle school and high school students in the district are now back for a few hours of in-person instruction per week, and elementary schoolers are back only 2.5 hours four days per week.

It will take years, even decades, to fully measure and understand the harms of prolonged school closures on children. But preliminary data from California’s Bay Area, where public schools were not and are still not fully open, show increased visits to emergency rooms for children in mental health crises, and increased suicidality and eating disorders. Nationally, data on massive learning losses, weight gain, unreported child abuse, and children missing from their school districts are extremely worrisome.

While influences on school closures are multifactorial, it is reasonable to conclude, based on the number of times this model was cited by district officials and school board members, that incorrect predictions create unfounded hesitancy in policymakers. Numerous examples of wildly inaccurate Covid-19 models exist, some summarized in a March 2021 report in The Lancet Microbe.

Conduct simple analyses of publicly available data to inform policy

Inaccurate models converging with media narratives create the second factor skewing Covid-19 school reopening policies: the related ideas that variants of SARS-CoV-2 are causing children to be sicker than previous strains, that cases and hospitalizations are increasing in children more than in older people, and that these phenomena are driving new surges across ages.

Much of the misinformation around variants is gleaned from popular media reports, although much of it originates with Covid-19 experts. Local news and smaller media outlets usually lack in-house data departments, but bigger news outlets have no excuse for their failure to dig into publicly available data from the Centers for Disease Control and Prevention and state databases to unpack these narratives to see if they are accurate. If not, those narratives should be corrected.

Spoiler alert: They often are not accurate or corrected.

We did what we propose large media outlets should do: investigate narratives put forward by Covid-19 voices with large platforms to see if they were supported by data.

We used publicly available data collected by the CDC from Michigan and Minnesota, two states in the news throughout March and April due to their case surges, to examine claims that children were getting sicker due to variants and that cases and hospitalizations were rising in children more than in other ages.

Michigan and Minnesota, which both have a high prevalence of the B.1.1.7 variant, had large surges in cases between March and early April, but showed no increase in severe illness (represented by hospitalizations) among children in these states (Figure 1). This finding is consistent with two recent papers in Lancet Infectious Diseases and Lancet Public Health showing no evidence of increased virulence of the B.1.1.7 variant in the United Kingdom.

Weekly Covid-19 deaths Michigan Minnesota
ONE TIME USE Chart by Leslie Bienen, Eric Happel, and Monica Gandhi

Our analyses indicate that Covid-19 cases are not increasing more rapidly in school-aged children nationwide than in any other age group. Although K-12 aged children are the least-vaccinated age group, they had the lowest rise in cases except 18- to 24-year-olds.

Most importantly, the data invalidate the claim that hospitalizations were being led by younger patients. When we examined the CDC data for cases per 100,000 people, hospitalizations among those aged 5 years to 17 years barely budged and were still quite low, and hospitalizations among 18-49 rose very slightly (Figure 2). The biggest change was exactly what should have been expected given that older individuals were vaccinated first: The rise in hospitalizations was low and mostly among those over 50 who were not yet widely vaccinated.

U.S. weekly Covid-19 hospitalizations
Figure 2 Chart by Leslie Bienen, Eric Happel, and Monica Gandhi

Graphs of Michigan and Minnesota’s cases (Figure 3) show a similar story, with Michigan experiencing more of a surge. Starting with the trough in mid-February and ending April 8, school-aged children in Michigan had the smallest increases in cases of any age group except older vaccinated groups.

Michigan, Minnesota weekly Covid-19 cases
Figure 3 Chart by Leslie Bienen, Eric Happel, and Monica Gandhi

Despite more cases in Michigan, hospitalizations in the age 5 to 17 group were not up at all, except for a small blip in the week of March 20. Michigan began requiring asymptomatic testing in early March for sports participation, which may explain the higher number of cases identified. It is unlikely that a smaller proportion of children with Covid-19 in Michigan were suddenly requiring hospitalization, supporting the idea that Michigan’s surge was unlikely to be driven by children.

The hypothesis that young people are spreading Covid-19 to adults as asymptomatic carriers, but not themselves falling sick, is invalidated by studies showing that asymptomatic young people do not readily spread infection to family members.

Three simple solutions

We suggest three changes to create more rational Covid-19 school policies moving forward.

First, school decision-makers should stop relying on “predictive” models that are too often wrong and create unnecessary and harmful amounts of caution toward school reopening. Instead, modelers and media outlets should draw upon publicly available real-world data from the CDC and vetted state databases. Using inputs based on assumptions related to unquantifiable human behaviors, such as fear and fatigue, rather than quantifiable data based on local hospitalization rates, in-school transmission events, and vaccination rates has led to school policies based on emotions rather than data.

Second, much misinformation comes from experts fitting data to preconceived narratives rather than the other way around. Although this is difficult to stop, large media outlets with data bureaus could and should investigate these narratives before repeating and amplifying them. They should also ensure they are using best data practices such as comparing case and hospitalization numbers corrected for population distribution. Cherry-picking date ranges to make data fit a narrative should be avoided.

As we demonstrated by following these simple recommendations, there was no surge in hospitalizations among school children, and cases in young people in Michigan and Minnesota were occurring at the same proportion, or slightly less often, than in other age groups.

Third, school policymakers should stop using case rates as a barometer of Covid-19 fluctuations and focus exclusively on hospitalizations and deaths. Guidance from the Oregon Department of Education, for example, currently suggests that schools in small counties return to all remote learning when cases exceed 90 per 100,000 people, regardless of the demographics of cases. Case numbers, however, are a highly variable statistic since they depend on factors such as testing rates or the presence of symptoms. Sickness and death are more important and reliable outcomes than case rates. As the least healthy and oldest people get vaccinated, case rates and hospitalizations will become further uncoupled. This is already happening. Although about half of U.S. states are seeing increases in cases, deaths are not rising steeply in nearly all states, but are rising slightly, holding steady, or declining.

A trio of researchers recently suggested a population-based metric of 5 hospitalizations per 100,000 people as a cutoff to remove all school-based restrictions such as wearing face coverings and physical distancing. The CDC and state health departments should develop a series of reliable metrics to guide lifting restrictions on schools and youth sports.

These changes will help policymakers create better, more effective policies to guide school reopening. We owe this much to children, whose lives have been severely upended during this yearlong pandemic.

Leslie Bienen is a veterinarian, scientific editor, and professor of public health at the OHSU-Portland State School of Public Health. Eric Happel is director of global strategy, capabilities, and excellence for Nike stores. Monica Gandhi is an infectious disease physician, professor of medicine, and associate division chief of the division of HIV, infectious diseases, and global medicine at UCSF/San Francisco General Hospital. Beinen and Happel are affiliated with ED300, an organization that advocates for opening Oregon schools and sports.

Source: STAT