Researchers try to connect elementary school kids masking to Covid+ wastewater
A crappy study, pardon the pun
Sometimes I worry that the entire field of medical science is sinking. Two weeks ago, Cochrane published an updated analysis of medical masks to prevent the spread of respiratory viruses in community and health care settings, and it was entirely null.
The CDC director herself dismissed it as ‘only including RCTs’— but uhh yeah, that’s called a strength.
Cochrane is the best of what we can do in medicine.
Now, the internet is abuzz with a new abstract. Researchers show an association between increased mask wearing and a lower chance of having + covid PCR from an elementary school’s wastewater in the next 5 days. (I paste the full abstract below), but here is the key bit. 60 observations, 9 days with any + effluent, and a CI that goes right up to .99 (Of course!)
Here are the issues
There are only 9 cases we are trying to predict with log regression. Some might call that thin data. It is very thin, and all estimates incredibly noisy (see width of CI)
The methods can’t separate 3 kids with Covid vs. 1000 kids with covid. It’s either wastewater + or negative.
The study appears significant (but just barely) after adjustment — was this plan registered? The concern is obviously for p hacking.
Kids who don’t urinate or deficate at school may also have COVID. (spoiler alert). Including these cases may change the result.
What happens if 10 different people observe a school, what is the kappa coefficient for % masking. Is that known? Probably it varies widely. I suspect the exposure is basically a random number.
Were any kids actually sick? The sewage is less likely to set off the PCR machine, but are any actual human outcomes changed? Kids staff?
Did the investigators use the toilets at school? This fact must be known. The Heisenberg uncertainty principle of pooping applies. Or did researchers poop and urinate only at the end of the day?
Why just 5 days? How many days before you start excreting sars-cov-2? Perhaps we are missing some cases!
NONE OF THIS PROVES ANYONE GOT COVID IN SCHOOLS
I don’t want to belabor this. This is low quality evidence. It uses a dubious endpoint, and a dubious exposure variable to create a dubious link with thin, sparce data. The CI is just *barely* significant with adjustment.
Why are we even talking about this?
You could prove ivermectin works with this methodology. You could prove prayer works or amulets. This is not how we test medical policies.
Excuse the pun: but this research is crappy.
PS another reminder AAP is awful
Brandolinis Law reigns supreme, as the Gish Gallop continues.
You just spent more effort debunking this (literal) shit than the authors put forth crafting it.
I'm not a medical expert or an expert in statistical analysis... In fact probably not an expert in anything even remotely close to being relevant. However even I can see this study is so short on controls and long on all sorts of possible and unaccounted for variables. I did teach school for a number of years...and I know a lot of things could impact the source data. For instance... What snacks were brought from home and what was served in the lunchroom that may have impacted the amount of bodily eliminations on the test days? What activities were going on in classrooms that may have impacted the number of kids going to the bathroom at school versus waiting until the end of the day or until home? Or, even more potentially impactful on the conclusions, what is the normal fluctuation of the virus found in the wastewater over time with no masking or with full masking? I'm amazed that anyone would look at that and feel proud and that they did a good job. Doesn't make sense.