My enthusiasm for Chat GPT in medicine is down

A few months ago, I wrote a few pieces on ChatGPT and it's role in medicine. I am still hopeful that it will someday play a role in medical communication with patients, including answering patients questions (see the work of John Ayers and colleagues). But this is what has happened since.

I paid for a subscription to ChatGPT and have tried to use it for research purposes. Ultimately, I canceled my subscription because I could not see much use for it.

If you ask ChatGPT to summarize data, the software occasionally inserted false information in tables or figures. That error rate is unacceptable for anyone doing academic research. References were sometimes false as well.

Next, I tasked one student who works with me to turn one of my presentations into a paper. He took the video of the talk, had Chat GPT write a transcript and summarize it into 3000 words and then he worked off that draft. Ultimately, though the paper was so reworked that I am not sure it retained any similarity to the draft and I am not sure it saved him time had he summarized it himself.

ChatGPT can draft letters of rec, but again these are reworked so much that it is of little use. ChatGPT can't draft original essays because by the time you tell it what you want, you have already drafted the essay.

We have one project where we need to code subjective attitudes. Here chatgpt does help largely because the software cannot be accused of having a bias to conclude anything.

In my clinical practice, LLMs are no where to be found and have not changed my day to day.

Overall, I haven't seen much opportunity for incorporating ChatGPT. What do you think? How do you use it?