Working Paper Finds COVID-19 Lockdowns Had Little Public Health Effect and Draws Criticism of Being Flawed

Three economists have concluded in a John Hopkins working paper that nonpharmaceutical interventions (NPIs) in response to the first wave of the COVID-19 pandemic did not have a significant effect on mortality rates.

Lockdowns in the Europe and U.S. reduced COVID-19 mortality by 0.2% on average, concluded Steve Hanke, a professor of applied economics at Johns Hopkins; Jonas Herby, a special advisor at the Center for Political Studies in Copenhagen; and Lars Jonung, professor emeritus in economics at Lund University in Sweden.

The working paper was published on the website of the Johns Hopkins Institute for Applied Economics, Global Health, and the Study of Business Enterprise. As a working paper, it has not gone through peer reviewed. Epidemiologists and public health experts have criticized the statistical methods and underlying definitions it used.

White House Press Secretary Jen Psaki was asked about the paper at today’s press briefing. She deferred to the administration’s experts to address the specifics of the paper, but she added that “the president has been clear that we are not pushing lockdowns, we have not been pro-lockdown, that has not been his agenda. Most of the lockdowns actually happened under the previous president. What our objective has been is to convey we have the tools we need to keep our country open.”

The findings from the Hopkins paper ricocheted around the media and on social media the past couple of days. Conservative outlets coverage the findings; mainstream media outlets did not.

The working paper consist of a review 34 studies and a meta-analysis that included 24 of them. In addition to concluding that lockdowns produced limited results on COVID-19-related deaths, Herby, Jonung and Hanke’s found that shelter-in-place orders reduced mortality from COVID-19 by 2.9%. However, in their conclusion they noted that closing nonessential businesses “seems to have had some effect,” reducing COVID-19 mortality by 10.6%, and that reduction was probably related to closing bars. 

Some of the most pointed criticism of the meta-analysis is about the definitions used. The studies in the meta-analysis were grouped into three categories: lockdown stringency index studies, shelter-in-place order (SIPO) studies, and specific NPI studies.

In a statement posted on Science Media Centre, a British clearinghouse of information for science and health journalists, Neil Ferguson, OBE, FMedSci, an epidemiologist and director of the MRC Centre for Global Infectious Disease Analysis, Jameel Institute, Imperial College London, noted that lockdown policies varied dramatically among countries, a range that makes defining the term a challenge. Ferguson also criticized the statistical methods used in the meta-analysis for not taking into account the lagging effects of lockdowns on reduced transmission and deaths. Growth rates in cases or deaths, with appropriate time lags, not total deaths, are the NPI outcomes that should be measured, Ferguson said in the posted statement.

Sami Bhatt, professor statistics and public health at Imperial College London, also commented on the issue of what gets categorized as a lockdown and urged caution when interpreting the findings.

“The authors define lockdown as ‘as the imposition of at least one compulsory, non-pharmaceutical intervention,’” Bhatt said in a statement posted on the same website as Ferguson’s. “This would make a mask wearing policy a lockdown. For a meta-analysis using a definition that is at odds with the dictionary definition (a state of isolation or restricted access instituted as a security measure) is strange.”

Both Bhatt and Ferguson have authored previous studies on the impact of lockdowns and NPIs on COVID-19 transmission and mortality. One used a model simulation and predicted a lockdown would significantly reduce mortality. The other evaluated interventions in 11 European countries and found NPI had “a large effect on reducing transmission” with an estimated 4% or less of the population in the countries infected with the virus.

Herby, Jonung and Hanke’s meta-analysis included studies establishing a relationship between mortality and lockdown policies. They excluded studies evaluating the effect of voluntary behavioral changes and recommendations, as well as studies using cases, hospitalizations, or other measures; interrupted time series studies comparing mortality rates before and after lockdowns; studies analyzing the effect of early vs later lockdowns; and synthetic control method papers.

Their analysis looked at specific NPIs. Of the NPIs studied, face mask mandates had a large effect of reducing mortality (–21.2%), although this was based on only two studies, and one looked specifically at an employee mask mandate. They found that school closures reduced mortality buy 4.4%. However, lockdowns and limited gatherings actually modestly increased mortality (0.6% and 1.6%, respectively). “Although this appears to be counterintuitive, it could be the result of an (asymptomatic) infected person being isolated at home under a SIPO can infect family members with a higher viral load causing more severe illness,” wrote Herby, Jonung and Hanke.

Border closures/quarantine led to just a 0.1% reduction. However, the studies on border closures were mixed with two studies reporting mortality reductions (15.6% and 24.6%) and another, a a large increase (36.3%).

The economist mooted some explanations for the limited effect (according to their meta-analysis): public appetite was low for following costly disease prevention efforts, especially when transmission was low; mandates were not able to regulate all areas of life where people could possibly become infected; and unintended consequences — such as isolating an asymptomatic person at home where they can infect others — played a larger role than expected.

“While this meta-analysis concludes that lockdowns have had little to no public health effects, they have imposed enormous economic and social costs where they have been adopted,” Herby, Jonung and Hanke concluded. “In consequence, lockdown policies are ill-founded and should be rejected as a pandemic policy instrument.”