ON
← Back to feed
How accurate are AI chatbots when we ask them a medical question?
Ireland🏛️ Politics5 days ago

How accurate are AI chatbots when we ask them a medical question?

An article from The Irish Times examines the accuracy of AI chatbots when answering medical questions. Researchers tested five major chatbots—ChatGPT, Gemini, Grok, Meta AI, and DeepSeek—with 50 health-related questions, finding that nearly a fifth of responses were highly problematic, half were problematic, and 30% were somewhat problematic. Performance varied by question type, with open-ended questions being more challenging. None of the chatbots provided fully accurate reference lists. A separate study showed that users relying on chatbots were less effective at identifying correct diagnoses and determining appropriate care compared to those using traditional methods. However, when chatbots were used without human interaction, their diagnostic accuracy improved significantly.

Go to the primary sources (2)

The official sources this coverage is built on. Read them directly to bypass framing.

1 reports

The Irish Times logoThe Irish TimesIndependent🔒Center5 days ago
How accurate are AI chatbots when we ask them a medical question?

An article from The Irish Times examines the accuracy of AI chatbots when answering medical questions. Researchers tested five major chatbots—ChatGPT, Gemini, Grok, Meta AI, and DeepSeek—with 50 health-related questions, finding that nearly a fifth of responses were highly problematic, half were problematic, and 30% were somewhat problematic. Performance varied by question type, with open-ended questions being more challenging. None of the chatbots provided fully accurate reference lists. A separate study showed that users relying on chatbots were less effective at identifying correct diagnoses and determining appropriate care compared to those using traditional methods. However, when chatbots were used without human interaction, their diagnostic accuracy improved significantly.

Bias read (Center): The article presents findings from multiple studies without overtly favoring any particular political stance. While it highlights concerns about AI reliability in healthcare—a topic with potential regulatory and ethical implications—it does not take a clear ideological position. The focus remains on

Keep the news honest.

ObjectiveNews is reader-funded and ad-free — we show you the bias instead of hiding it. Support independent journalism for €5/month.

Become a Supporter

Related stories