Groundbreaking BBC research shows issues with over half the AI assistants tested
Conducted over a month, the study saw the BBC test four prominent, publicly available AI assistants – OpenAI’s ChatGPT; Microsoft’s Copilot; Google’s Gemini; and Perplexity. These AI assistants were given access to the BBC’s website and asked questions about the news, prompting them to use BBC News articles as sources where possible. AI answers were reviewed by BBC journalists, all experts in the question topics, on criteria including accuracy, impartiality and how they represented BBC content.
Pete Archer, Programme Director for Generative AI at the BBC says: “We’re excited about the future of AI and the value it can bring audiences. We have already used it to add subtitles to programmes on BBC Sounds and translate content into different languages on BBC News. AI can bring real value if used responsibly.”
“But AI is also bringing significant challenges for audiences. People may think they can trust what they’re reading from these AI assistants, but this research shows they can produce responses to questions about key news events that are distorted, factually incorrect or misleading. The use of AI assistants will grow so it's critical the information they provide audiences is accurate and trustworthy.”
“Publishers, like the BBC, should have control over whether and how their content is used and AI companies should show how assistants process news along with the scale and scope of errors and inaccuracies they produce. This will require strong partnerships between AI and media companies and new ways of working that put the audience first and maximise value for all. The BBC is open and willing to work closely with partners to do this.”
Issues identified in AI responses
Some examples of the significant problems identified in responses from these AI assistants include:
- Factual errors
- Misrepresentation of source material
The full research can be found on the BBC website.
Read more: Article from CEO, BBC News and Current Affairs, Deborah Turness
