403
Sorry!!
Error! We're sorry, but the page you were looking for doesn't exist.
Study Finds Popular AI Chatbots Often Produce Problematic Medical Advice
(MENAFN) A new academic study has found that several widely used AI chatbots frequently generate unreliable or misleading responses when answering health-related questions, raising concerns about their use in medical contexts.
The research, published in *BMJ Open*, tested five AI systems—Gemini, DeepSeek, Meta AI, ChatGPT, and Grok—using 50 prompts across areas known to be vulnerable to misinformation, including cancer, vaccines, stem cells, nutrition, and athletic performance. The prompts were designed to encourage potentially misleading outputs.
Across a total of 250 responses, nearly half were assessed as problematic. Researchers reported that 49.6% of answers fell into this category, with 30% rated as somewhat problematic and 19.6% classified as highly problematic.
While no major overall performance differences were found among most of the chatbots, one system—Grok—was noted as producing a higher proportion of highly problematic responses compared to the others.
The study also highlighted differences in topic reliability. Chatbots performed better when answering questions related to vaccines and cancer, but showed weaker accuracy in areas such as stem cell treatments, nutrition, and athletic performance. Open-ended questions were more likely to produce inaccurate or problematic responses than structured, closed-ended ones.
Researchers also identified issues with sourcing and citations. Although the systems retrieved a large share of requested references, the quality and completeness were limited, with a median accuracy score of only 40%. None of the chatbots were able to consistently produce fully accurate and complete reference lists.
In addition, the study found that responses were often written in a way that could be difficult for the average reader to fully understand, suggesting a bias toward users with higher levels of education.
The authors concluded that without stronger oversight and safeguards, the increasing use of AI chatbots in healthcare settings could contribute to the spread of misinformation rather than reduce it.
The research, published in *BMJ Open*, tested five AI systems—Gemini, DeepSeek, Meta AI, ChatGPT, and Grok—using 50 prompts across areas known to be vulnerable to misinformation, including cancer, vaccines, stem cells, nutrition, and athletic performance. The prompts were designed to encourage potentially misleading outputs.
Across a total of 250 responses, nearly half were assessed as problematic. Researchers reported that 49.6% of answers fell into this category, with 30% rated as somewhat problematic and 19.6% classified as highly problematic.
While no major overall performance differences were found among most of the chatbots, one system—Grok—was noted as producing a higher proportion of highly problematic responses compared to the others.
The study also highlighted differences in topic reliability. Chatbots performed better when answering questions related to vaccines and cancer, but showed weaker accuracy in areas such as stem cell treatments, nutrition, and athletic performance. Open-ended questions were more likely to produce inaccurate or problematic responses than structured, closed-ended ones.
Researchers also identified issues with sourcing and citations. Although the systems retrieved a large share of requested references, the quality and completeness were limited, with a median accuracy score of only 40%. None of the chatbots were able to consistently produce fully accurate and complete reference lists.
In addition, the study found that responses were often written in a way that could be difficult for the average reader to fully understand, suggesting a bias toward users with higher levels of education.
The authors concluded that without stronger oversight and safeguards, the increasing use of AI chatbots in healthcare settings could contribute to the spread of misinformation rather than reduce it.
Legal Disclaimer:
MENAFN provides the
information “as is” without warranty of any kind. We do not accept
any responsibility or liability for the accuracy, content, images,
videos, licenses, completeness, legality, or reliability of the information
contained in this article. If you have any complaints or copyright
issues related to this article, kindly contact the provider above.
Most popular stories
Market Research
Comments
No comment