Can you trust AI such as ChatGPT with health questions?
Move over Dr Google: users are turning to artificial intelligence chatbots such as ChatGPT to answer medical questions. However, can you trust AI like you can a GP? Almost a third of us (31%) hope that AI can improve our access to healthcare, according to the Office for National Statistics (ONS). In addition, AI has a huge part to play in healthcare thanks to its abilities to spot patterns – for instance, in X-ray scans – and the government is already investing millions in AI for the NHS.
Reliability of AI for Medical Questions
Previous research has suggested that up to 39.5% of healthcare professionals hope to rely on large language models (LLMs) such as ChatGPT to make medical decisions. But new research suggests that consumers should be wary of relying on bots for big health decisions and diagnoses.
The researchers found that AI software such as ChatGPT and Google’s Bard – now Gemini – are potentially inaccurate. Posing 56 common medical questions to the chatbots, the answers were reviewed by two doctors. The answers offered by the AI bots were often either inaccurate or incomplete.
With ChatGPT, just over a quarter of answers were accurate (28.6%) while 28.6% were inaccurate and 42.8% were partially accurate.
Expert Opinion
Dr. Andrei Brateanu, of the Cleveland Clinic Foundation in the US, said the study showed that AI tools shouldn’t be used as a substitute for doctors. "They can be considered as additional resources that, when combined with human expertise, can enhance the overall quality of information provided," he said.
Rather than LLMs such as ChatGPT, other AI tools are built to be used by clinicians to speed up diagnosis – instead of being a ‘substitute’ for flesh-and-blood doctors.
Government Investment in AI for Healthcare
In the UK, the government has invested £21m to help NHS staff diagnose and treat patients with strokes, cancers, and heart conditions more quickly. Staff are being equipped with decision-making tools which use AI technology to improve diagnosis.
Another tool uses AI to analyze chest X-rays, the tool most commonly used to diagnose lung cancer – helping clinicians to deal with the 600,000 chest X-rays performed each month in England.
The government has previously said: "Artificial intelligence is already transforming the way we deliver healthcare and AI tools are already making a significant impact across the NHS in diagnosing conditions earlier, meaning people can be treated more quickly."
Sources: ChatGPT is easy to fool – and might be less useful than we thought (Yahoo News UK), AI researcher calls for 'immediate worldwide ban' (Business Insider), AI researcher says '10-20%' chance AI will take over (Fortune)
