ChatGPT and Google Gemini are terrible at summarizing news ...
An unsurprising result
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Study Finds Inaccuracies in AI Chatbot Summarizations
A new study from the BBC has found that four of the world's most popular AI chatbots including ChatGPT are inaccurately summarizing news stories.
The study compared the summarization accuracy of ChatGPT, Copilot, Gemini, and Perplexity by summarizing 100 news stories from the BBC.
Key Findings of the Study
The study revealed that a significant percentage of AI answers had various issues, with factual errors present in a notable number of responses. In particular, inaccuracies were found in the summarizations provided by Gemini, ChatGPT, and Copilot.
Examples of inaccuracies highlighted in the study include Gemini's misrepresentation of NHS recommendations and ChatGPT/Copilot erroneously referencing politicians who were no longer in office.
Furthermore, the AI chatbots struggled to distinguish between opinions and facts, often editorialized content, and lacked essential context in their summaries.
Implications and Recommendations
The study's results emphasize the challenges in utilizing AI chatbots for accurate news summarization. Despite advancements in AI technology, issues persist in distinguishing between factual information and opinions.
The BBC concluded that Copilot by Microsoft and Gemini by Google exhibited more significant inaccuracies compared to ChatGPT by OpenAI and Perplexity.
It is highlighted that skepticism towards AI summary tools is justified, and caution should be exercised when relying on AI-generated news summaries.
While AI technology continues to evolve rapidly, improvements in accuracy are necessary to enhance the reliability of AI chatbot summarizations.
Conclusion
As AI progresses at a pace faster than Moore’s law, advancements in AI software are inevitable. However, until AI chatbots can consistently provide accurate and reliable news summaries, it is advisable to approach AI-generated content with discernment.
For the latest news and tech updates, it's recommended to rely on trusted sources like TechRadar for accurate and verified information.
