Randomized trials are the BEST way to test masking & other NPIs
Anyone who says otherwise is wrong
A recent article in STAT has made medical experts groan. Rather than accept Cochrane's meta-analysis of RCTs of masking, the authors wish to abandon randomization entirely. Unfortunately, the essay is riddled with basic misunderstandings about the role of randomized trials. One epidemiologist texted me, “Every paragraph of that article hurt my soul.” Here I will explain why it is problematic.
Of course, it should come as no surprise that the article is being amplified by Eric Topol. In a series of posts here, I have described instances where Dr. Topol has promoted incorrect conclusions from academic papers. Given the harms of incorrect inferences, I remain surprised the media quotes him so often. Here he endorses the title that ‘the last thing you’d do to prove masks help protect against covid is a randomized trial’
Turning to the essay. It begins by assuming the very thing it set out to prove: face masks and distancing were initially the “only ways to prevent transmission of the virus”. That is what we are debating, not a fact we agree upon.
The essay then praises the scientific community but notes “it could not produce studies with desired surety”. This is misleading. The US and UK and WHO and CDC failed to conduct even 1 cluster RCT of masking. Experts went on TV and exaggerated or distorted the evidence, praising cloth masks for years, while failing to run even a single study. Saying we ‘could not produce studies’— should read— we ‘did not try’.
Anthony Fauci is particularly culpable as he controlled US research funding. Had he followed science (which states run experiments when you don’t know), he would have commissioned multiple RCTs. He didn’t, and instead just changed his rhetoric, while Cochrane remained unchanged.
The authors are aware that observational studies--they highlight a Lancet paper-- are hopelessly confounded, and admit as much, and they seem aware of the two recent Cochrane meta-analyses. But this is where their logic train starts to derail completely.
The authors insinuate that all available RCTs were done incorrectly. They claim “RCTs have value only when researchers can be sure the treatment is administered as intended”. This is wrong. Compliance with an intervention is part of the recipe. If participants find masking burdensome and cheat by not wearing it perfectly--- that actually does mean masks don't work. A diet trial of “don't eat anything” would fail, and that would not be a failure of the RCT method, but proof the advice was bad. Lars Hemkins, an evidence expert, puts it simply:
In addition to faulting imperfect adherence in the masking arms of RCTs, the authors fault contamination of the control arm (People on control arm wearing masks). Here, their understanding is also limited. Contamination of the control arm can be measured in RCTs (it was measured in Bangladesh), and is not inherently a deal breaker.
They appear unaware of many trials with modest crossover that were nevertheless highly informative, e.g. Courage & Determination, where 20% of the control arm ultimately got the intervention, but both trials prove that routine upfront use is not warranted.
Nor are they aware of methods to correct for contamination (instrumental variables). This is a basic tool of epidemiology.
The authors claim RCTs of behavioral interventions may just produce “noise”. Their ignorance of decades of behavioral RCTs which provide useful information is staggering.
To build the case for masks, the authors cite “strong evidence” from laboratory results, theoretical analyses and RCTs of health care workers.
Laboratory results confuses bio-plausibility with evidence. Theoretical analyses confuses modeling with data. Models are often wrong (See also: COVID19). And RCTs of health care workers does not link to the RCTs-- so I cannot verify their claim-- but even if it were true, an 8 year old in school or 40 year old in a Home Depot would be an inappropriate extrapolation of health care worker data. Having said that, I think their reference is wrong or misstated.
The authors call for carefully designed “field trials”. If they mean pragmatic RCTs I would agree. Somehow I doubt that.
The authors make the bizarre argument that we should not run field trials if researchers cannot show they will “help, rather than harm”. Here they seem to be saying that if the trial won't give them their preferred positive conclusion, they would not want it run. This fits with their general bias that the only good evidence is one that agrees with their position, but is not scientific. We run RCTs precisely because we don’t know what they will show.
The Stat authors call RCTs “fatally flawed”. They urge researchers not to perform research that “cannot answer the question they ask”. But the question I am asking is: 1. Do community mask mandates slow spread? 2. Do masks work in children? 3. If so, at what ages? 4. Do mask recommendations help? 5. If someone is compliant with masking, and you randomize them to continue or discontinue are outcomes better a year later? (This tests the case of perfect use). All of these questions are best answered with RCTs
In 2021, I wrote an article calling for RCTs with many arms to assess all these questions:
Instead the question these authors wish to answer is “which study gives me the answer I desire”. Well that study design is called: an observational study with massive analytic flexibility, which is what they seem to prefer.
Overall the essay reflects a sophomoric understanding of RCTs. The biggest masking failure was not running more RCTs, under a variety of circumstances, in a variety of locations and settings. This failure falls squarely on leaders at NIAID, CDC, WHO, and UK NHS. But continuing the practice for 3 years, after vaccination, that failure falls on all of us.
In the current biomedical system the best way to get people who believe things work to prove that they work is to say: RCT or we are going to stop doing this. Given that 3 years have passed and mask proponents have failed to show benefit in RCT, masks need to go the same place we put hydroxychloroquine and ivermectin— the trash heap— but not the ocean.
Vinay, you are too nice. Topol (with whom we both have personal interaction) is an idiot -- through and through. His knowledge level on many topics (including epidemiology) is way BELOW sophomoric. A kindergartener would be ashamed of most of the things he says. It is perhaps one of the saddest commentaries on medicine that he holds the position he does. Scripps stopped getting all referrals from me when he took that spot. He has done them no favors.
This article is on my list for worst top 10 articles over the past five years (and there have been so many bad ones, that is a hard place to be). The premises are wrong; the reasoning is wrong; the conclusions are actually dangerous.
You are usually frank, but this deserves much harsher treatment than you gave it. Articles like this do us all a world of damage.
The clear purpose of the STAT article is to make the case for not doing studies that might not support already-in-place public policy. That’s it; there is no science in the article at all. As the article admits, “Such research sows confusion that erodes trust in science, misleads policymakers, depletes social capital, and squanders critical resources. We believe that many of these studies should never have been done at all, reserving resources for studies that could improve health outcomes… Such studies can confuse people who want to know how effective face masks are, while emboldening people who are already completely convinced that face masks are ineffective — and are looking for grounds to sow doubt about them.“ So that’s the point… any nuance in the results of a study, that don’t fully support a policy in place, should not be published, because nuance “sows confusion” and “erodes trust in science.” Just have “experts” tell people what to do, and suppress anything that doesn’t provide absolute evidence for existing policy.