University of Waterloo - Education News Canada
A team of researchers based at the University of Waterloo have created a new tool - nicknamed "RAGE" - that reveals where large language models (LLMs) like ChatGPT are getting their information and whether that information can be trusted. LLMs like ChatGPT rely on "unsupervised deep learning," making connections and absorbing information from across the internet in ways that can be difficult for their programmers and users to decipher. Furthermore, LLMs are prone to "hallucination" - that is, they write convincingly about concepts and sources that are either incorrect or nonexistent.
"You can't necessarily trust an LLM to explain itself," said Joel Rorseth, a Waterloo computer science PhD student and lead author on the study. "It might provide explanations or citations that it has also made up." Rorseth's team's new tool employs a recently developed strategy, called "retrieval-augmented generation" (RAG), to understand the context for LLMs' answers to a given prompt. "RAG allows users to provide their own sources to an LLM for context. Our tool illustrates how different sources lead to different answers when using RAG, helping to assess whether that information is trustworthy," Rorseth said. Because their tool focuses on retrieval-augmented generation explainability, they nicknamed it "RAGE' against the machine."
Understanding where LLMs like ChatGPT are getting their information - and ensuring they're not repeating misinformation - will only become more important as highly sensitive, human-centered industries like the medical and legal sectors adopt these tools, Rorseth said. "We're in a place right now where innovation has outpaced regulation," he said. "People are using these technologies without understanding their potential risks, so we need to make sure these products are safe, trustworthy, and reliable." The research, "RAGE Against the Machine: Retrieval-Augmented LLM Explanations", was published in the proceedings of the 40th IEEE International Conference on Data Engineering.
Education News Canada
Education News Canada is part of the Jaguar Media Group. Our daily e-newsletter delivers the latest news and developments related to the education field. In addition to publishing news issued by universities, colleges, school boards, governments, and related organizations, we also conduct a thorough press review coming from Canada’s daily newspapers and over 400 regional and local news sources.
Jaguar Media Group also publishes:
