AI chatbots can’t summarize news accurately, says BBC study
The British Broadcasting Corporation’s latest study reveals that AI chatbots distort and mislead when they report the news.
Deborah Turness, the CEO of BBC News and Current Affairs, wrote in a blog that AI brings “endless opportunities.”
However, companies developing AI tools were “playing with fire.”
Article continues after this advertisement
READ: Encouraging the world to pay for online news
“How long will it be before an AI-distorted headline causes significant real-world harm?” she asked.
AI chatbots fall short in news reporting
A BBC study finds 51% of AI answers to questions about the news had “significant issues”, and 19% of answers citing BBC content introduced factual errors (@matthew_weaver / The Guardian)https://t.co/e9M9a1Zn5fhttps://t.co/CP0CYln4h9
— Mediagazer (@mediagazer) February 11, 2025
The BBC gave the popular AI chatbots ChatGPT, Copilot, Google Gemini and Perplexity AI-generated content from the BBC website.
Article continues after this advertisement
Then, it asked them questions about the news.
As a result, the AI bots provided answers with “significant inaccuracies” and distortions.
For example, Google Gemini mistakenly stated that “The NHS advises people not to start vaping and recommends that smokers who want to quit should use other methods.”
The NHS does not recommend vaping as a way to quit smoking.
Moreover, the BBC study found these insights regarding AI news reporting:
- 51% of all AI chatbots’ answers to news questions had significant issues.
- 19% of AI answers citing BBC had factual errors like incorrect statements, dates and numbers.
- 13% of BBC article quotes had alterations from the original or were absent in the article cited.
BBC chief Deborah Turness says “AI assistants do not discern between facts and opinion in news coverage.”
Also, they don’t distinguish between current and archive material and tend to add opinions into their answers.
Nevertheless, Pete Archer, BBC Program Director for Generative AI, expects more people to use AI assistants.
That’s why they should provide audiences with accurate and trustworthy information.
The BBC and other publications should have control over how AI companies use their content.
The tech and media sectors must work together to ensure people can find trustworthy information in the age of AI.