Target trials are a failed methodology

A recent target trial is contradicted by a randomized study in 2 days.

The concordance between observational studies and randomized trials on the same clinical question has long been discussed. A quarter century ago in the NEJM paired papers show that the two designs often, but not always, agree.

The problem is often is not good enough. A coin can give you 50%— you need ~100% agreement before you can trust an observational study design. In an effort to achieve this concordance, many modifications have been tried. Propensity score matching, inverse probability weighting, and dubious natural experiments have all been attempted. These efforts have yielded concordance of 60-80%, which frankly isn’t good enough— because you don’t know the 20-40% of the time you are wrong. If you are willing to accept 20-40% discordance— then let’s celebrate p values of 0.05-0.4 in randomized studies.

Enter Target trials. Surely they will solve the problem. Here it the latest installment in the saga.

A target trial to assess if statins reduce or increase dementia; The only thing more exciting than a target trial of vitamin D.

Achieves a huge win.

What’s the problem? A 9000 person randomized trial appearing literally days after this paper in the NEJM completely contradicts it.

(I discuss this paper on sensible medicine this week).

I love how the target trial authors end with a call for an RCT, and a pertinent one appears 2 days later. They could have taken the semester off.

What is the lesson?

I think it is time to recognize that target trials are looking like they are merely observational studies rebranded. These studies needed to have extremely high predictive value to succeed, and already have fallen short. They have fallen short on very important questions like multi-billion dollar drugs given to millions. They are unlikely to supplant RCTs.

With advances with AI, I suspect it won’t be long before AI itself recommends randomization over target trial emulation— that’s when I know for sure, it will have surpassed most researchers.