I feel sympathy to Nature Medicine editors. The editorial staff could use editors with better medical and statistical backgrounds. Perhaps they lack the funding for these hires. The journal famously published a time of immunotherapy RCT from China, which appeared fraudulent a mile away— they had to retract that one.
They are back with a new paper on the Shingles vaccine making yet more magical claims. Shall we examine the work?
The premise is that this trial is a ‘natural experiment’
The idea is simple: compare the period of time in the first gray box to the second, where the type of vaccine used was dramatically different. Natural experiment means the people in the first and second gray box are otherwise comparable— they both got a shingles shot— the only difference is they are staggered 12 months or so in time.
To assess for the quality of a natural experiment, look at table 1. The two groups should be similar…
Holy cow, that’s not the case.
More women, younger age, more blacks, less asians, more married people— these are totally different groups of people.
This is not a natural experiment.
So, to fix this little screw up, the authors have to do propensity score matching. But that just makes this all a propensity score matched observational study with one problem— there is a temporal covariate imbalance! You might as well have just run a clean PS observational study.
Here is the main result
The authors spin this to mean the new vaccine lowers the event rate, but the alternative explanation was the old vaccine raised the event rate, or that ***MOST LIKELY** there is residual confounding because these are different groups of people.
Also look how fast the effect is. Take heart failure. How does a vaccine reduce cumulative heart failure 100 days after receipt? Heart failure is the destination of a longstanding maladaptive process, it doesn’t happen in an instant— its not twisting your ankle, it is getting a hip replacement.
This study is as low quality as all other shingles literature, which I have detailed here:
Shingles vaccine(s) and dementia: the only medical intervention to work faster than Tums
I am going to talk about how fast shingles vaccines reverse dementia across 4 papers. Turns out the effect is near instant. But before I do that, let us agree on some facts about the nature of the evidence used to make these claims.
Final thought, I don’t understand why vaccine researchers are competing with fringe groups in the battle of who can produce the most misinformation.
If I were trying to get people to believe vaccine science, I would stop publishing false, misleading or low quality work, making outrageous claims. If someone wants to get or not get a Shingles shot— that decision should be based on…. spoiler alert… Shingles. Why do you need to produce false research trying to show dementia reductions or cardiovascular event reductions? I don’t understand why the push for clearly implausible findings—
Are you trying to get a job in the Cambridge sociology department or as President of Stanford or something?
If you like this stack, become a subscriber
Having just yesterday got part two of my shingles vaccine I was doubly disappointed here. Apparently I am no less likely to get dementia and Dr Prasad won’t reveal the important bit. Is it good at preventing shingles? I had shingles about eight years ago and didn’t much enjoy it.
I would love an analysis on shingles…i think there are two types live and attenuated(?). 🙏