In what is arguably the most important decision the Food and Drug Administration has made this year — its emergency use authorization of the Pfizer/BioNTech Covid-19 vaccine — the agency apparently assigned only a single reviewer in each of two key scientific disciplines (clinical and statistics) to do the work in three weeks that usually takes months to do.
The FDA’s authorization last week followed similar authorizations in the United Kingdom and Canada. But the FDA’s decision is particularly important because of its reputation for being the international “gold standard” in regulatory rigor.
Unlike its counterparts in other countries, the FDA is believed to be the only drug regulator in the world that consistently receives and reviews patient-level data from the clinical trials that underpin drug and vaccine approvals. To perform such rigorous analyses, the FDA typically spends around 10 months (a mere six months for applications given “priority review” designation) in an effort that involves reviews by experts representing various scientific disciplines: clinical medicine, statistics, pharmacology, chemistry, pharmacovigilance, and more. Together, these reviews form an “action package” which, by law, must be made publicly available 30 days after approval.
Given the urgency of the pandemic, the review of the Pfizer/BioNTech vaccine was conducted far faster than usual. The centerpiece of the analysis was data from the company’s 44,000-participant Phase 3 trial. FDA reviewers had just three weeks, from Nov. 20 to Dec. 11, to complete their analyses. It was a monumental task, which raises the question: Why didn’t the FDA devote additional reviewers to it? According to the FDA’s review memo, some scientific disciplines, such as pharmacovigilance, had multiple reviewers involved. But the two disciplines tasked with examining the clinical trial data and results, the clinical and statistical reviewers, were seemingly left to do their work solo.
This seems wholly inadequate on at least two levels. First, without additional reviewers it is hard to comprehend how the work of several months could be squeezed into a matter of 22 days (including Saturdays and Sundays). In-depth review calls for examining patient-level data — a large feat that involves auditing and reviewing individual case records as well as independently rerunning analyses on the raw data.
Before the pandemic, it was typical to see just one clinical reviewer’s name for any given application. But given the stakes — and the time crunch — involved with reviewing Covid-19 vaccines, we would have thought the agency would do an even more thorough job than normal. But that does not appear to be the case.
One of us (P.D.) raised questions about potential unblinding in the trials through the vaccine’s side effects, as well as about the confounding effects of fever- and pain-reducing medications, which participants in the vaccine arm took three to four times more often than those in the placebo arm. Yet the FDA’s review shows no evidence that any of its scientists investigated either of these issues, and without more scientific staff devoted to the task it is hard to imagine how they could.
As one of us (M.H.) has investigated in the past with respect to FDA approvals not related to Covid-19, disagreements within the agency about whether to approve an intervention or set limitations upon its use are relatively common during the review process. Among 174 approvals examined between 2011 and 2015, 42 (24%) contained at least one disagreement among reviewers.
A key takeaway from this research is that a real strength of the FDA is its capacity to entertain dissent with a view to making better judgments about complex scientific evidence.
Such differing opinions were on display at last week’s FDA advisory committee meeting, where four advisers voted against the emergency use authorization that the FDA ultimately granted the next day. But that was the FDA’s external advisory committee.
Whether such disagreement also exists within the FDA is unclear. By assigning only one clinical, one statistical, and one toxicology scientist to review the Pfizer/BioNTech Covid-19 vaccine, it seems that discussion, let alone disagreement, was curtailed by design. The pressure on those lone FDA reviewers to do their work in record time, and do it without raising serious questions about the data, was likely immense.
Having an official second or third reviewer for all core scientific disciplines would have helped. Not making more reviewers available strikes us as an effective way to prevent in-depth assessment of the underlying data and limit the possibility of dissent. Shortcuts in the regulatory process undermine the very purpose of regulation to protect the public. If the goal was speed at all costs, we should just get rid of regulators. Otherwise, they need all the resources they can get to do their job, including the fast-approaching decision about whether to authorize Moderna’s Covid-19 vaccine.
Peter Doshi leads the Restoring Invisible and Abandoned Trials initiative and is an associate professor of pharmaceutical health services research at the University of Maryland School of Pharmacy. Matthew Herder is director of the Health Law Institute at the Schulich School of Law and an associate professor of pharmacology in the faculty of medicine, both at Dalhousie University in Canada.
Competing interests: Doshi has received grants from the FDA, and the Laura and John Arnold Foundation, and is an unpaid member of the Reagan-Udall Foundation for the FDA. Herder is a member of the Patented Medicine Prices Review Board, Canada’s national drug price regulator, and holds grants from the Canadian Institutes of Health Research.