Academic economics was in the same place in the mid 80s. Fancy econometric models were used to “fix” non-random assignment in observational studies. In 1986 the American Economic Review published a paper by Robert Lalonde where he took known RCT results and pretended they were observational and applied the econometric “fixes”, with disastrous results. This paper was massively influential and spawned a generation of economists who refused to just throw more math at observational studies but instead focused on finding natural experiments. Folks like Anrgist, Card, Krueger, Levitt. That’s now the standard approach taught to undergrads.
Hopefully RCT-DUPLICATE has the same effect - but unlike in economics, medicine has pharma and the press clamoring for spurious results so we shall see.
"...spawned a generation of economists who refused to just throw more math at observational studies but instead focused on finding natural experiments."
I'm no stats expert, but the idea that mathematical adjustments can fix bad data seems fundamentally flawed to me. I remember being taught in my intro stats classes that no amount of data manipulation can overcome a bad dataset.
And thank you for the Lalonde reference - I'll have to go look that up.
It’s so simple, right? But it isn’t actually bad data. It’s just that observational studies suffer from unobserved confounders. The math generally involves fancy ways of using observables to proxy for those unobserveds and justifying them with asymptotic arguments. The intuition is that this should get you closer to the truth. But
A) maybe, this isn’t uniformly true with multiple confounders, and
"Bad" was probably not the best word to use in my original comment. Inappropriate or unsuitable would be more accurate. The idea that you can correct for unobserved confounders by using math sounds good in theory, but the obvious problem is that this implies that you know what the correct values should be! That's the whole reason why randomization is so powerful if it's done properly, because it controls for both known and unknown confounders, not just the ones that the researcher thinks are important. If the dataset you have isn't quite fit for purpose, you need to get a proper dataset, not manipulate the one you have. It feels like the classic case of trying to fit a square peg into a round hole.
Any level III evidence that allows causal language is scary, but I didn't get to vote. When I found out you could use "increased/reduced" the risk to describe the results of a TTE, I thought Oh Boy, here we go. These studies are Level III evidence even if prospective. Because they are NOT randomized. This makes me crazy. At least in the oseltamivir study they used "associated with an adjusted risk reduction of 1.8%." But your average resident is going to cite that study & say Soanddo et al demonstrated that oseltamivir reduced the risk of in-hospital mortality by 1.8%," claiming more than the authors did.
The problem is not target trial emulation per se, but the poor methodology often implemented under the "target trial" label. As a result, the literature is increasingly populated with contradictory and sometimes uninformative target trial studies. In many cases, this reflects the combination of easy access to large healthcare databases and inadequate training in causal inference, epidemiologic study design, and bias assessment.
Calling a study a target trial does not automatically make it valid, just as calling a 1979 Chevy a Ferrari does not make it a Ferrari. Likewise, a poorly designed randomized controlled trial still remains vulnerable to bias simply because it is labeled an RCT.
At the end of the day, many clinically important questions are simply not amenable to randomization. For these questions, target trial emulation may be the best available approach. Moreover, for every target trial that fails to replicate an RCT, one can point to others that closely corroborate randomized evidence. The key issue is therefore not the
method itself, but the rigor with which it is designed, executed, and interpreted.
I.will agree.with this proposal, but only if everyone also agrees to rename "pragmatic trials" to become "experiments where the treatment intervention is too weak to ever have the slightest chance of influencing outcomes.". Obviously the new name is still a work in progress.
I dint hear of such a thing as target trials before. I looked up what it was and it's a wishful thinking method to try to re-invent observational (surveys, clinical notes/data) as a stand-in for randomized controlled trials.
When I say wishful thinking it is a crock invented for the lazy, the deceitful, or both.
Dr Prasad exposed the target trial nonsense in a nice way.
Just have to watch the movie Idiocracy to see where we're headed into the future. Shameful when considering the achievements of the past and the extreme power at our fingertips that we devolve so rapidly. It seems striving for answers is closely aligned with struggling to survive.
Academic economics was in the same place in the mid 80s. Fancy econometric models were used to “fix” non-random assignment in observational studies. In 1986 the American Economic Review published a paper by Robert Lalonde where he took known RCT results and pretended they were observational and applied the econometric “fixes”, with disastrous results. This paper was massively influential and spawned a generation of economists who refused to just throw more math at observational studies but instead focused on finding natural experiments. Folks like Anrgist, Card, Krueger, Levitt. That’s now the standard approach taught to undergrads.
Hopefully RCT-DUPLICATE has the same effect - but unlike in economics, medicine has pharma and the press clamoring for spurious results so we shall see.
"...spawned a generation of economists who refused to just throw more math at observational studies but instead focused on finding natural experiments."
I'm no stats expert, but the idea that mathematical adjustments can fix bad data seems fundamentally flawed to me. I remember being taught in my intro stats classes that no amount of data manipulation can overcome a bad dataset.
And thank you for the Lalonde reference - I'll have to go look that up.
It’s so simple, right? But it isn’t actually bad data. It’s just that observational studies suffer from unobserved confounders. The math generally involves fancy ways of using observables to proxy for those unobserveds and justifying them with asymptotic arguments. The intuition is that this should get you closer to the truth. But
A) maybe, this isn’t uniformly true with multiple confounders, and
B) “closer” is not synonymous with “close”
"Bad" was probably not the best word to use in my original comment. Inappropriate or unsuitable would be more accurate. The idea that you can correct for unobserved confounders by using math sounds good in theory, but the obvious problem is that this implies that you know what the correct values should be! That's the whole reason why randomization is so powerful if it's done properly, because it controls for both known and unknown confounders, not just the ones that the researcher thinks are important. If the dataset you have isn't quite fit for purpose, you need to get a proper dataset, not manipulate the one you have. It feels like the classic case of trying to fit a square peg into a round hole.
I always appreciate your insights and thoughts. Thank you.
Thank you for speaking the truth, Vinay!!! That is rare in the medical profession.
Any level III evidence that allows causal language is scary, but I didn't get to vote. When I found out you could use "increased/reduced" the risk to describe the results of a TTE, I thought Oh Boy, here we go. These studies are Level III evidence even if prospective. Because they are NOT randomized. This makes me crazy. At least in the oseltamivir study they used "associated with an adjusted risk reduction of 1.8%." But your average resident is going to cite that study & say Soanddo et al demonstrated that oseltamivir reduced the risk of in-hospital mortality by 1.8%," claiming more than the authors did.
Middle authors who didn't help! I died. Hey, they provided resources!
The problem is not target trial emulation per se, but the poor methodology often implemented under the "target trial" label. As a result, the literature is increasingly populated with contradictory and sometimes uninformative target trial studies. In many cases, this reflects the combination of easy access to large healthcare databases and inadequate training in causal inference, epidemiologic study design, and bias assessment.
Calling a study a target trial does not automatically make it valid, just as calling a 1979 Chevy a Ferrari does not make it a Ferrari. Likewise, a poorly designed randomized controlled trial still remains vulnerable to bias simply because it is labeled an RCT.
At the end of the day, many clinically important questions are simply not amenable to randomization. For these questions, target trial emulation may be the best available approach. Moreover, for every target trial that fails to replicate an RCT, one can point to others that closely corroborate randomized evidence. The key issue is therefore not the
method itself, but the rigor with which it is designed, executed, and interpreted.
I.will agree.with this proposal, but only if everyone also agrees to rename "pragmatic trials" to become "experiments where the treatment intervention is too weak to ever have the slightest chance of influencing outcomes.". Obviously the new name is still a work in progress.
Glad you are back doing this! Whatever you did at the corrupt government agency was invisible to me. This I can use.
It’s probably time to rename pharmaceutical research as very expensive highly credentialed marketing papers.
Building AI models for studies affords biases in the study itself. We’ve already lived through biased modeling schemes.
Thanks
I dint hear of such a thing as target trials before. I looked up what it was and it's a wishful thinking method to try to re-invent observational (surveys, clinical notes/data) as a stand-in for randomized controlled trials.
When I say wishful thinking it is a crock invented for the lazy, the deceitful, or both.
Dr Prasad exposed the target trial nonsense in a nice way.
Observational studies are in general poorly done; they nearly always lack a negative control or two.
TTE has its own set of biases.
Just have to watch the movie Idiocracy to see where we're headed into the future. Shameful when considering the achievements of the past and the extreme power at our fingertips that we devolve so rapidly. It seems striving for answers is closely aligned with struggling to survive.
He's got his edge,
It's Ockham's razor
♥️
🦘🌏🪃
https://x.com/i/grok/share/980fe995cc304d24845df78a905302c4