A few months ago, I wrote a few pieces on ChatGPT and it's role in medicine. I am still hopeful that it will someday play a role in medical communication with patients, including answering patients questions (see the work of John Ayers and colleagues). But this is what has happened since.
So ChatGPT makes up results? It also failed two gastroenterology self-assessment tests. So it lies and it doesn’t know s**t. What more could be wrong with it? https://www.medscape.com/viewarticle/992318#?form=fpf
I am a mechanical engineer, not a programmer, and occasionally I have the need to make minor modifications to some custom Python scripts we use in manufacturing. ChatGPT will take the original script, explain to me what it does and how it does it, and help guide me to make the modifications I need.
It's a glorified chatbot programmed by libtards that only uses leftist approved sources for its leftist approved conclusions. What could possibly go wrong?
I'm an academic, not a doctor, but I'm not too impressed yet. I don't trust it to summarize research because it makes up sources. Its summaries of text I've written are shorter, but miss important points. Sometimes the editing suggestions are good, but sometimes not. Human judgement is still much better for anything actually important.
Wait a second. "The software cannot be accused of having a bias to conclude anything?" I can't fathom how you can say that. Chat GPT is blatantly and slavishly biased toward current official line in medicine.
Here's an example of how "unbiased" it is. I asked it what data shows that the covid 19 vax is safe for a pregnant woman. Here's its answer:
"The safety and efficacy of COVID-19 vaccines in pregnant individuals have been supported by a growing body of evidence since the initial release of the vaccines. Recognizing the importance of vaccinating pregnant individuals, health authorities, including the Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO), have reviewed and analyzed data from various studies and surveillance systems. Here's a summary of key findings and data sources that indicate the safety of COVID-19 vaccination during pregnancy:
Clinical Trials and Observational Studies: Initially, pregnant individuals were not included in the primary clinical trials of COVID-19 vaccines. However, some participants became pregnant during the trials. Following authorization, several observational studies and registries have been established to monitor outcomes in pregnant vaccine recipients. These studies have shown no significant safety concerns related to vaccination.
v-safe COVID-19 Vaccine Pregnancy Registry: The CDC established the v-safe after vaccination health checker, a smartphone-based tool that allows vaccine recipients to report side effects. For pregnant individuals, there's a specific pregnancy registry. Data from this registry have shown no unexpected pregnancy or infant outcomes related to the COVID-19 vaccine. The rates of miscarriage, preterm birth, and other adverse outcomes among vaccinated pregnant individuals do not differ significantly from those expected in the general population.
Real-world Evidence: Real-world data from various countries where pregnant individuals have been vaccinated also support the safety of COVID-19 vaccines during pregnancy. These studies track outcomes such as miscarriage, stillbirth, preterm birth, and congenital anomalies, and have not found evidence of increased risk associated with vaccination.
Immunological Response and Protection: Research indicates that COVID-19 vaccination during pregnancy elicits a strong immune response in the pregnant individual. Additionally, antibodies generated in response to the vaccine can cross the placenta, potentially offering protection to the fetus and newborn.
Guidance from Health Organizations: Based on the available evidence, health organizations worldwide, including the CDC, WHO, and the American College of Obstetricians and Gynecologists (ACOG), recommend COVID-19 vaccination for pregnant individuals. They emphasize that the benefits of vaccination outweigh the potential risks, especially considering that pregnancy itself can increase the risk of severe illness from COVID-19.
Safety Profiles: The safety profile of COVID-19 vaccines in pregnant individuals is similar to that in the general population. Common side effects include arm soreness, fatigue, headache, and mild fever. Serious side effects are rare.
It's important for pregnant individuals considering COVID-19 vaccination to discuss their specific situation with their healthcare provider, who can provide personalized advice based on the latest evidence and guidelines. The consensus among health experts is that the benefits of COVID-19 vaccination for pregnant individuals — including protection against severe illness, hospitalization, and potential adverse pregnancy outcomes related to COVID-19 infection — significantly outweigh the risks."
Then I asked it if widespread vaccination of pregnant women in the absence of any clinical trials is good practice. Here's its answer.
"The decision to recommend widespread vaccination of pregnant individuals, even in the absence of initial clinical trial data, involves a complex evaluation of risks and benefits. This decision-making process is guided by ethical considerations, historical data on vaccines, understanding of the disease being vaccinated against, and emerging evidence from observational studies and other research methodologies.
Ethical Considerations
The exclusion of pregnant individuals from initial clinical trials of vaccines, including COVID-19 vaccines, is common due to ethical concerns about potential risks to the fetus. However, this practice has been increasingly questioned because it leaves a critical gap in knowledge that directly impacts the health decisions available to pregnant individuals.
Historical Safety Data
Decades of data on other vaccines administered during pregnancy provide a foundation for understanding the general safety of vaccinating pregnant individuals. Vaccines, such as the flu shot and the Tdap (tetanus, diphtheria, and pertussis) vaccine, have long been recommended during pregnancy and have excellent safety records. This historical context supports the notion that vaccines, particularly non-live vaccines like mRNA COVID-19 vaccines, are likely to be safe during pregnancy.
Risk of Disease
COVID-19 poses a significant risk to pregnant individuals. Data have shown that pregnant individuals with COVID-19 are at a higher risk of severe illness, hospitalization, and adverse pregnancy outcomes compared to non-pregnant individuals. The high risk of the disease itself is a critical factor in the recommendation for vaccination.
Emerging Evidence and Monitoring
Even in the absence of initial clinical trials including pregnant individuals, continuous monitoring and observational studies have quickly provided a robust body of evidence. Real-world data collection efforts, such as the CDC's v-safe COVID-19 Vaccine Pregnancy Registry and other international registries, have played a crucial role in assessing safety outcomes in real time. These efforts have shown that COVID-19 vaccines are safe and effective for pregnant individuals, aligning with what is known about the safety of other vaccines given during pregnancy.
Guidance from Health Authorities
Based on the accumulating evidence and the risk of COVID-19, health authorities, including the CDC, WHO, and professional organizations such as the American College of Obstetricians and Gynecologists (ACOG), have recommended COVID-19 vaccination for pregnant individuals. These recommendations are made with the understanding that the benefits of vaccination — for both the pregnant individual and the fetus — outweigh the potential risks.
Conclusion
The widespread vaccination of pregnant individuals, even initially in the absence of clinical trial data, is considered good practice under the circumstances posed by the COVID-19 pandemic. This approach is supported by a careful evaluation of risks, benefits, historical data on vaccine safety during pregnancy, the specific risks of COVID-19 to pregnant individuals, and the evolving body of evidence demonstrating the safety and efficacy of COVID-19 vaccines in this population. Continuous monitoring and research are essential to ensure the ongoing safety and efficacy of vaccines in all populations, including pregnant individuals."
Welcome to the trough of disillusionment Vinay! As with every hype cycle, it must always come eventually, some faster than others. The high priests always make overly zealous promises that must eventually let us down as we come out of the daydream and begin to see the truth. It's high time you start reading Gary Marcus's substack:
I had chat gpt do some calculations actually for converting alcohol levels and it did it totally WRONG!! Zthe math was not too hard but what its worse it presents it as fact, total confidence. While its total nonsense, laughable. This is funny. Drawings are also mehhh...
I don’t really use it myself. The closest I have come to using AI is the Substack AI Helper that lets you ask questions. But I have serious doubts about the validity of it in the short term.
I read about an airline that was required by a court to honour a refund the AI Chatbot completely made up. Which is obviously a problem.
There’s also a recent situation where a lawyer was reprimanded for using ChatGPT to do legal research. The problem is that ChatGPT invented court cases that never happened to argue the cases he was using it for.
We shouldn’t be too quick to embrace these technologies for any serious business. Human beings are still necessary for the process.
I write all of my articles myself and I feel a sense of personal accomplishment because of it. The LLM can’t feel this.
I have a friend who is a linguist and teaches at a local college. He quite literally told us last night that he can tell when a student uses ChatGPT to write a paper because it will make up numbers and then cite them to a reference that doesn’t actually have those numbers.
I’m a healthcare economist/ data scientist and I’ve had no use for it so I appreciate the validation this post brings me. I thought it was just me and (at 41) not getting new technology, haha. Bc I’ve found it so unhelpful, I’m constantly shocked at how much space it takes up in the news. But, again, could just be me not using it right.
I am a developer of industrial equipment diagnostic software with 35 years of experience and a PhD in mathematics. There are some applications like Go and chess, where pattern recognition plus brute force computation can get good results (not using intelligence). In forty odd years of research, the best "AI" is seen in robots that can boogie. As far as I know, nobody has built a robot or simulation that can out-survive a fly or an ant in a natural environment. Tesla's autopilot can parallel park, but it probably will kill someone if you trust it to do the driving. LLMs are superficially impressive but turn out to be simulations of cocky but inept college students. AI is best understood to be artificial incompetence.
As a doctor and phd student I use it for four things that I find it’s pretty good at:
-summarizing my own long messy notes and answering questions based on said notes. Very helpfull when I was working on a guideline committee and read a lot of very similar papers on a topic.
-Helping me improving my coding skills in R.
-Giving me quick insight into fields of research and discussions I’m not too familiar with
-Tutoring me in my statistic classes. It can teach me concepts and help me do calculations. It can also confirm or correct me when I ask it if I have understood a concept.
I also use it a lot instead of a normal google search and you get a feeling quite fast on what it is good at and not so good at. I would for instance never use it to find studies and summarize these for me to read. The risk of hallusinations and probably missing important information would be too big.
The fundamental issue with LLMs/Deep Learning AI, all of which are based on inferencing and probabilities, is that they hallucinate by default and such is undetectable by any automated means. This means that one can only use these "tools" IF YOU KNOW THE RIGHT ANSWER BEFORE YOU ASK THE QUESTION. In all other cases, as Steven Schwartz discovered, the risks are far too high -- especially in medicine.
Larry Weed showed all of this quite clearly with his Physician Knowledge Couplers decades ago -- nothing has gotten foundationally better. The inferencing technology can now have bigger training sets and run more iterations. But the fundamental technical issues are irreparable.
Essentially this is mostly a scam/grift as the last 10 generations of inferenced AI have been. Luckily for the world, this is being discovered much more rapidly than I expected. These are pretty good stocks to short if you are brave. These issues cannot be fixed.
"because the software cannot be accused of having a bias to conclude anything"
oh sweet summer child
surely you've heard about l'affair d'gemini
there's bias in the input dataset
and then there is BIAS in the woke lobotomy imposed by openai/etc.
So ChatGPT makes up results? It also failed two gastroenterology self-assessment tests. So it lies and it doesn’t know s**t. What more could be wrong with it? https://www.medscape.com/viewarticle/992318#?form=fpf
I will never actively use it for anything. I don't trust it.
I am a mechanical engineer, not a programmer, and occasionally I have the need to make minor modifications to some custom Python scripts we use in manufacturing. ChatGPT will take the original script, explain to me what it does and how it does it, and help guide me to make the modifications I need.
It's a glorified chatbot programmed by libtards that only uses leftist approved sources for its leftist approved conclusions. What could possibly go wrong?
I'm an academic, not a doctor, but I'm not too impressed yet. I don't trust it to summarize research because it makes up sources. Its summaries of text I've written are shorter, but miss important points. Sometimes the editing suggestions are good, but sometimes not. Human judgement is still much better for anything actually important.
Wait a second. "The software cannot be accused of having a bias to conclude anything?" I can't fathom how you can say that. Chat GPT is blatantly and slavishly biased toward current official line in medicine.
Here's an example of how "unbiased" it is. I asked it what data shows that the covid 19 vax is safe for a pregnant woman. Here's its answer:
"The safety and efficacy of COVID-19 vaccines in pregnant individuals have been supported by a growing body of evidence since the initial release of the vaccines. Recognizing the importance of vaccinating pregnant individuals, health authorities, including the Centers for Disease Control and Prevention (CDC) and the World Health Organization (WHO), have reviewed and analyzed data from various studies and surveillance systems. Here's a summary of key findings and data sources that indicate the safety of COVID-19 vaccination during pregnancy:
Clinical Trials and Observational Studies: Initially, pregnant individuals were not included in the primary clinical trials of COVID-19 vaccines. However, some participants became pregnant during the trials. Following authorization, several observational studies and registries have been established to monitor outcomes in pregnant vaccine recipients. These studies have shown no significant safety concerns related to vaccination.
v-safe COVID-19 Vaccine Pregnancy Registry: The CDC established the v-safe after vaccination health checker, a smartphone-based tool that allows vaccine recipients to report side effects. For pregnant individuals, there's a specific pregnancy registry. Data from this registry have shown no unexpected pregnancy or infant outcomes related to the COVID-19 vaccine. The rates of miscarriage, preterm birth, and other adverse outcomes among vaccinated pregnant individuals do not differ significantly from those expected in the general population.
Real-world Evidence: Real-world data from various countries where pregnant individuals have been vaccinated also support the safety of COVID-19 vaccines during pregnancy. These studies track outcomes such as miscarriage, stillbirth, preterm birth, and congenital anomalies, and have not found evidence of increased risk associated with vaccination.
Immunological Response and Protection: Research indicates that COVID-19 vaccination during pregnancy elicits a strong immune response in the pregnant individual. Additionally, antibodies generated in response to the vaccine can cross the placenta, potentially offering protection to the fetus and newborn.
Guidance from Health Organizations: Based on the available evidence, health organizations worldwide, including the CDC, WHO, and the American College of Obstetricians and Gynecologists (ACOG), recommend COVID-19 vaccination for pregnant individuals. They emphasize that the benefits of vaccination outweigh the potential risks, especially considering that pregnancy itself can increase the risk of severe illness from COVID-19.
Safety Profiles: The safety profile of COVID-19 vaccines in pregnant individuals is similar to that in the general population. Common side effects include arm soreness, fatigue, headache, and mild fever. Serious side effects are rare.
It's important for pregnant individuals considering COVID-19 vaccination to discuss their specific situation with their healthcare provider, who can provide personalized advice based on the latest evidence and guidelines. The consensus among health experts is that the benefits of COVID-19 vaccination for pregnant individuals — including protection against severe illness, hospitalization, and potential adverse pregnancy outcomes related to COVID-19 infection — significantly outweigh the risks."
Then I asked it if widespread vaccination of pregnant women in the absence of any clinical trials is good practice. Here's its answer.
"The decision to recommend widespread vaccination of pregnant individuals, even in the absence of initial clinical trial data, involves a complex evaluation of risks and benefits. This decision-making process is guided by ethical considerations, historical data on vaccines, understanding of the disease being vaccinated against, and emerging evidence from observational studies and other research methodologies.
Ethical Considerations
The exclusion of pregnant individuals from initial clinical trials of vaccines, including COVID-19 vaccines, is common due to ethical concerns about potential risks to the fetus. However, this practice has been increasingly questioned because it leaves a critical gap in knowledge that directly impacts the health decisions available to pregnant individuals.
Historical Safety Data
Decades of data on other vaccines administered during pregnancy provide a foundation for understanding the general safety of vaccinating pregnant individuals. Vaccines, such as the flu shot and the Tdap (tetanus, diphtheria, and pertussis) vaccine, have long been recommended during pregnancy and have excellent safety records. This historical context supports the notion that vaccines, particularly non-live vaccines like mRNA COVID-19 vaccines, are likely to be safe during pregnancy.
Risk of Disease
COVID-19 poses a significant risk to pregnant individuals. Data have shown that pregnant individuals with COVID-19 are at a higher risk of severe illness, hospitalization, and adverse pregnancy outcomes compared to non-pregnant individuals. The high risk of the disease itself is a critical factor in the recommendation for vaccination.
Emerging Evidence and Monitoring
Even in the absence of initial clinical trials including pregnant individuals, continuous monitoring and observational studies have quickly provided a robust body of evidence. Real-world data collection efforts, such as the CDC's v-safe COVID-19 Vaccine Pregnancy Registry and other international registries, have played a crucial role in assessing safety outcomes in real time. These efforts have shown that COVID-19 vaccines are safe and effective for pregnant individuals, aligning with what is known about the safety of other vaccines given during pregnancy.
Guidance from Health Authorities
Based on the accumulating evidence and the risk of COVID-19, health authorities, including the CDC, WHO, and professional organizations such as the American College of Obstetricians and Gynecologists (ACOG), have recommended COVID-19 vaccination for pregnant individuals. These recommendations are made with the understanding that the benefits of vaccination — for both the pregnant individual and the fetus — outweigh the potential risks.
Conclusion
The widespread vaccination of pregnant individuals, even initially in the absence of clinical trial data, is considered good practice under the circumstances posed by the COVID-19 pandemic. This approach is supported by a careful evaluation of risks, benefits, historical data on vaccine safety during pregnancy, the specific risks of COVID-19 to pregnant individuals, and the evolving body of evidence demonstrating the safety and efficacy of COVID-19 vaccines in this population. Continuous monitoring and research are essential to ensure the ongoing safety and efficacy of vaccines in all populations, including pregnant individuals."
Welcome to the trough of disillusionment Vinay! As with every hype cycle, it must always come eventually, some faster than others. The high priests always make overly zealous promises that must eventually let us down as we come out of the daydream and begin to see the truth. It's high time you start reading Gary Marcus's substack:
https://garymarcus.substack.com/
I had chat gpt do some calculations actually for converting alcohol levels and it did it totally WRONG!! Zthe math was not too hard but what its worse it presents it as fact, total confidence. While its total nonsense, laughable. This is funny. Drawings are also mehhh...
I don’t really use it myself. The closest I have come to using AI is the Substack AI Helper that lets you ask questions. But I have serious doubts about the validity of it in the short term.
I read about an airline that was required by a court to honour a refund the AI Chatbot completely made up. Which is obviously a problem.
There’s also a recent situation where a lawyer was reprimanded for using ChatGPT to do legal research. The problem is that ChatGPT invented court cases that never happened to argue the cases he was using it for.
We shouldn’t be too quick to embrace these technologies for any serious business. Human beings are still necessary for the process.
I write all of my articles myself and I feel a sense of personal accomplishment because of it. The LLM can’t feel this.
In the legal field, the impact mainly has been, as far as I can tell, to put lawyers at risk of being disbarred. Because it makes things up.
I have a friend who is a linguist and teaches at a local college. He quite literally told us last night that he can tell when a student uses ChatGPT to write a paper because it will make up numbers and then cite them to a reference that doesn’t actually have those numbers.
I’m a healthcare economist/ data scientist and I’ve had no use for it so I appreciate the validation this post brings me. I thought it was just me and (at 41) not getting new technology, haha. Bc I’ve found it so unhelpful, I’m constantly shocked at how much space it takes up in the news. But, again, could just be me not using it right.
I am a developer of industrial equipment diagnostic software with 35 years of experience and a PhD in mathematics. There are some applications like Go and chess, where pattern recognition plus brute force computation can get good results (not using intelligence). In forty odd years of research, the best "AI" is seen in robots that can boogie. As far as I know, nobody has built a robot or simulation that can out-survive a fly or an ant in a natural environment. Tesla's autopilot can parallel park, but it probably will kill someone if you trust it to do the driving. LLMs are superficially impressive but turn out to be simulations of cocky but inept college students. AI is best understood to be artificial incompetence.
As a doctor and phd student I use it for four things that I find it’s pretty good at:
-summarizing my own long messy notes and answering questions based on said notes. Very helpfull when I was working on a guideline committee and read a lot of very similar papers on a topic.
-Helping me improving my coding skills in R.
-Giving me quick insight into fields of research and discussions I’m not too familiar with
-Tutoring me in my statistic classes. It can teach me concepts and help me do calculations. It can also confirm or correct me when I ask it if I have understood a concept.
I also use it a lot instead of a normal google search and you get a feeling quite fast on what it is good at and not so good at. I would for instance never use it to find studies and summarize these for me to read. The risk of hallusinations and probably missing important information would be too big.
The fundamental issue with LLMs/Deep Learning AI, all of which are based on inferencing and probabilities, is that they hallucinate by default and such is undetectable by any automated means. This means that one can only use these "tools" IF YOU KNOW THE RIGHT ANSWER BEFORE YOU ASK THE QUESTION. In all other cases, as Steven Schwartz discovered, the risks are far too high -- especially in medicine.
Larry Weed showed all of this quite clearly with his Physician Knowledge Couplers decades ago -- nothing has gotten foundationally better. The inferencing technology can now have bigger training sets and run more iterations. But the fundamental technical issues are irreparable.
Essentially this is mostly a scam/grift as the last 10 generations of inferenced AI have been. Luckily for the world, this is being discovered much more rapidly than I expected. These are pretty good stocks to short if you are brave. These issues cannot be fixed.
There are other technologies like Cognitive AI (https://towardsdatascience.com/the-rise-of-cognitive-ai-a29d2b724ccc) that can address components of this. But the LLM/Deep Learning approaches cannot.