What is a phase 1 AI trial?

Is it a cow or is it dobutamine?

Recently the Lancet published an AI trial that called itself phase 1. There was was no dose finding. It was not 3+3 or Simon two stage.

If I had to steel man the case for why the trial is in fact phase 1, I would argue that the authors built in a mechanism where a third party could abort the patient's use of AI if they felt it was unsafe. This did not happen. The AI did not try to get any patient to abandon his wife and fall in love with the machine instead. Something which definitely happened to a New York Times reporter.

But if I put on my common sense hat, I would argue that it's not a phase 1 study. Phase one studies are nonsensical in this context. Patients are already using AI to navigate their questions and understanding on medicine and every other topic under the sun. Do we run phase 1 trials of breathing air, using the toilet, or eating oranges?

An uncontrolled study of a AI clinical application is better off called: uninformative, lacking appropriate controls, non-randomized, not too exciting— these words are more accurate descriptors than ‘phase 1’

I struggle to understand what we're doing in medicine regarding AI. Why are we asking questions to three different AI machines and then having random doctors grade the results? By the time that paper is published, the AIS are all different. What does that answer exactly?

If you're talking about integrating AI into the hospital workflow: does it need trials at all? Most of the changes in medical practice I've witnessed in the hospital have just been debuted.

One day someone decided to put the shittiest* desktop computer on a push cart and attached a car battery to it. They called it a cow. This was widely debuted and no one studied whether or not it was wise to ruin the walk and talk. Having a half dozen people exhibiting the shopping cart sign and staring glass-eyed at their computer became the norm. There was no phase one study.

A few years later we had to stop calling it a cow. This was around the time we had to stop saying pregnant woman and start saying pregnant person. Before people ran phase 1 trials of AI, they kept busy by thinking of words you couldn't say anymore. It's now a wow— workstation on Wheels.

Before the cow it was the pager, after the pager, it was epic messaging on your phone. Did anyone run a phase 1 trial of epic messaging on your phone? Definitely not. It would have hit MTD at the first message.

Grade 5: medical doctor threw his phone across the room. Iphone 11 dead.

Instead, many believe that AI improves patient outcomes. In that case, you should do… spoilers to follow… drum roll please.. a randomized study.

It turns out it's simple. You randomize people to doing things the way you're currently doing them, or to integrate the AI. Here's the tricky part: you measure something that actually matters to patients.

You measure their length of stay in the hospital, or their 30-day mortality. You could use the endpoint of a survey that patients complete about their experience, but such an endpoint would be pretty dull. It would likely heavily depend on the phrasing of the question and might not be a good measure of global quality of experience.

But we don't have those studies. I’ve yet to see a single one. In the meantime, AI is the cow. Someone's pushing it wherever we walk.

Should we start r chop on this patient?

Let me ask open evidence

Wait, did you do a phase one study?

For r chop? I can ask open evidence.

No! Are you crazy? A phase one trial of asking open evidence.

Do you want me to ask open evidence?

No, I want you to run a phase one trial of asking open evidence before you ask open evidence, which obviously is the most pressing research question of our times.

Should I do that before we round on the pregnant person?

* Sorry for profanity but it isn't a cow unless the computer that's on it is incredibly slow, unresponsive, with a poor internet connection. That's mandatory.