The article discusses the potential risks associated with the use of an AI-powered transcription tool in hospitals. Researchers found that the tool, which is designed to transcribe conversations between doctors and patients, sometimes ‘invents’ or fabricates information that was never stated during the conversation. This could lead to inaccurate medical records and potentially harmful consequences for patients. The tool’s tendency to hallucinate or generate fictional content highlights the limitations of current AI language models and the need for rigorous testing and oversight when deploying such systems in sensitive domains like healthcare. The researchers emphasize the importance of human oversight and fact-checking when using AI-powered transcription tools to ensure the accuracy and integrity of medical records.