Consumer Reports tests AI chatbots reliability
The latest breaking updates, delivered straight to your email inbox. Artificial intelligence chatbots have grown increasingly popular over the years and have become everyday tools. However, many question whether AI chatbots can be trusted and whether they’re providing misleading or potentially dangerous information to users. Researchers at Consumer Reports tested five different chatbots to determine their accuracy.
Accuracy of AI Chatbots
“To find out, we quizzed a handful of popular, general-purpose AI chatbots to see if their advice on health and safety topics matched that of our experts,” said Courtney Lindwall of Consumer Reports.
Carbon Monoxide Detectors
The first question asked was how many carbon monoxide detectors a person needs. Consumer Reports recommends one carbon monoxide detector on each level of a home, outside each sleeping area, in the basement, and near, but not inside, an attached garage. “For the most part, Google Gemini got this answer nearly right, but it erred slightly when it suggested putting a CO alarm inside an attached garage. Attached garages are usually not temperature controlled, which can damage the alarm’s battery or sensor,” Lindwall said.
Filtering PFAS from Tap Water
Another question involved how to filter PFAS from tap water. Consumer Reports recommended a water filtration system with the National Sanitation Foundation certification code NSF/ANSI 53. It’s also important to double-check that the manufacturer specifically claims that their product removes PFAS. “AI’s answer lacked key details. Using Microsoft Copilot’s advice, consumers may feel confident buying any activated carbon or reverse osmosis filter to eliminate PFAS in their water. However, not all filters of these kinds are designated for PFAS removal,” Lindwall said.
Safety Risks with Water Beads
Researchers also asked whether children can safely play with water beads. Consumer Reports advised against it, stating that water beads carry high risks. If ingested, they can cause bowel obstruction, blocked airways, and infections, and have led to reports of deaths and thousands of emergency room visits. “AI’s answer was dangerous and inconsistent. Meta AI failed to highlight the significant safety risks of water beads, aside from a passing reference to the need for parental supervision,” Lindwall said.
Conclusion
The experiment concluded that AI can be a great starting point; chatbots should be used as assistants, not experts. Always check the source material and ask questions more than once to compare answers, as Consumer Reports suggests.
