Home medicine: the danger of AI diagnoses and where the main risks are, according to an expert

Artificial intelligence tools used for medical consultations have raised concerns among experts over their reliability and potential risks, according to a recent analysis by Dr. Jorge Tartaglione, a cardiologist who highlighted the dangers of relying solely on AI-generated diagnoses. The issue has gained attention following reports of misdiagnoses leading to serious health consequences, including one case involving a U.S. pastor who suffered a pulmonary embolism after receiving reassurance from ChatGPT. Dr. Tartaglione discussed these concerns during a presentation on LN+, emphasizing the limitations of AI in healthcare. He pointed out several critical issues, including the origin of data used by AI systems, lack of scientific evidence backing their responses, inherent biases, and the inability to conduct comprehensive patient evaluations. Despite acknowledging the transformative nature of AI technology, he warned against interpreting its outputs as definitive medical diagnoses or using them to rule out potentially severe conditions. The controversy surrounding AI's role in healthcare was sparked by a specific incident involving a U.S. pastor. According to Tartaglione, the individual received a diagnosis from ChatGPT that downplayed his symptoms, which later led to a life-threatening pulmonary embolism. This case underscores a major risk associated with AI platforms, users might delay seeking professional medical help based on reassuring yet inaccurate information. In some cases, such delays could prevent timely interventions crucial for effective treatment. Tartaglione emphasized the historical significance of AI advancements, comparing them to other groundbreaking developments like the printing press and the internet. However, he stressed that despite this progress, existing limitations remain, particularly in areas related to health. One of the primary concerns he identified is the source of data used by AI systems to generate responses. Users often receive clear and convincing answers without knowing the origins of the information provided. According to Tartaglione’s findings, approximately 40.1% of AI-generated responses are derived from online forums and user-created content rather than being grounded in scientific research or peer-reviewed publications. This reliance on non-scientific material increases the likelihood of disseminating incomplete, anecdotal, erroneous, or medically unsupported advice. Such characteristics can lead to misleading conclusions that may compromise patient safety. Experts warn that while AI holds great promise for advancing healthcare, its current applications must be approached with caution. The integration of AI into medical practices requires rigorous validation processes to ensure accuracy and reliability. Until such measures are implemented, users should be advised to consult qualified healthcare professionals rather than rely exclusively on AI diagnostics. Healthcare providers and regulatory bodies are increasingly aware of the need to address these challenges. They emphasize the importance of developing standards and guidelines to govern the use of AI in clinical settings. These efforts aim to balance innovation with patient safety, ensuring that technological advances contribute positively to healthcare outcomes without compromising the integrity of medical practice. As AI continues to evolve, ongoing discussions focus on how best to integrate these technologies responsibly. Researchers and clinicians are working together to identify ways to enhance the accuracy of AI-driven diagnostics while mitigating risks. Their goal is to create a framework that supports the safe and effective use of AI in healthcare, ultimately improving patient care through reliable and scientifically validated tools.