Science is still in its infancy: a few simple randomized trials could save us

We waste money on laboratory and observational research.

A few months ago, a study made the rounds that seemed to suggest that the timing of chemotherapy matters. If you got the first dose of immunotherapy in the earlier part of the day, you lived longer than people who got it later in the day.

Of course, the start time for intravenous infusional drugs is not randomly assigned. Like everything in life, there are more desired start times than others. And like everything in life, rich, well connected, motivated people get the better times.

Adding to the complexity: sometimes the patient seems so sick you want to expedite the first dose and you take any time you can get. For people who look like they are in no rush, you might wait for the preferred time.

All this is to say: there are likely unmeasured confounders associated with the timing of infusion. And it could be those confounders, and not the infusion timing itself, responsible for the differences.

Did the scientists adequately address for these factors? No. Not at all. They absolutely could not, because this is observational research. The granularity of the data did not permit it. Moreover, their rhetoric in the paper did not even seem to be aware of it.

Note: for those of you interested in this specific example, go look through the back catalog of plenary session podcast and you'll find it.

Yet, that did not stop the authors and field from uncritically parroting the conclusion, and pushing on this narrative. Timing matters.

Some scientists actually conduct research into mechanistic reasons why this might be the case. But the most obvious confirmatory study is something that no one is doing.

Take the next 1000 people and randomize their start times. Randomly assign the time, and see if there's any difference in outcome. You can take all cancers. You can take anybody—-there need be no inclusion criteria other than someone is scheduling for chemotherapy.

If you did this study, you'd have the gold standard answer in one month. Is there a benefit or not? And yet we don't do it.

I find it strange that a scientific enterprise would be happy to employ thousands of people to perform useless basic science, and useless observational studies, when a very simple randomized study would really settle the issue definitively.

If the study is positive, then look into the mechanism of action. But if the study is negative, there's no need to do your pipetting.

This is just an example, but the problem is everywhere in science. We will end this pandemic with zero randomized control trials of masking children. We will also end with 1,000 observational studies of masking children. Does this ratio make sense?

What sort of a society would pay for thousands of people to repeat a useless exercise, while simultaneously failing to perform the one very simple act that would definitely answer the question?

We will end the pandemic with numerous lawsuits about vaccine mandates for people who've already had COVID-19. We will also end with zero randomized studies that took people who already had COVID-19 and randomized them to vaccine dose(s) or no vaccine dose. We never did that study.

Does it make for a society to litigate this question in the courts, when it could be litigated in a study? If you randomize 10,000 people who've had COVID-19 to vaccine or not (we are only taking people who are interested in the study, we're not forcing anyone to join), you would definitively know if there is an additional benefit for getting the dose. Obviously, Pfizer has no incentive to do the study. It's up to the FDA to make them. But this FDA is so in Albert Bourla's pocket, they can tell us how much change he has in there. Very likely, when their terms are over they'll get some of that change.

Instead they want courts to settle the matter. I want to be clear, even if the study was positive, it doesn't mean you ought to mandate it, but if the study is negative then there's not even a prima facie case to mandate.

I strongly suspect it would be a negative study. I suspect it would be negative for hard outcomes like hospitalization and death. But I won’t be surprised if it's negative even for soft outcomes like mild URI symptoms and a positive test.

Many of us think that science in 2022 is at some peak. It's still on the upswing. And most of the mountain lies ahead of us. We have huge ways we can improve. We have turned many scientific arenas into a gold mine for useless careers when simple randomized studies could answer the question in less than 200 days if we set our mind to it.

I've heard some people lament public expenditure on art. But I want to be clear: I would rather find a 10 sculptors than the next ICD-10 VA database long covid study. I would rather fund six painters than a research team performing observational research about lockdowns or masking kids. I would rather fund 12 performance artists then one more laboratory studying the interaction between circadian rhythm and chemotherapy, before a definitive study shows there's a signal here worth chasing.

Science is still largely amateur hour. Most of what you read is untrue, or little more than a opinion poll. What concerns me is how few people truly understand the depth of this challenge, and how few have embraced the obvious solution: a few large simple randomized trials to elevate us from the darkness of ignorance.