How Artificial Intelligence Could Change Schools & Change How ...
Among other distinctions, Kristen DiCerbo can lay claim to being one of the first people on the planet to come face-to-face with the educational potential of generative artificial intelligence. In the fall of 2022, months before the public got a glimpse of ChatGPT, DiCerbo, a learning scientist and chief learning officer at Khan Academy, got access to a beta version of Open AI’s GPT-4 model. The startup needed Khan Academy’s help training it to pass the Advanced Placement biology exam, a requirement dreamed up by Microsoft co-founder Bill Gates, who wanted improved performance as a condition of handing Open AI more funding.
Khan Academy founder Sal Khan and DiCerbo negotiated a partnership with Open AI, and just five months later, their AI-powered Khanmigo tutoring bot debuted. Last summer, Khan Academy launched an AI writing coach. Nearly two years in, DiCerbo remains bullish on the possibilities of AI tutoring, cheerfully engaging critics about the limitations of the technology, even as by all measures it evolves and improves.
Challenges and Criticisms
Much of the press for Khanmigo has been positive: late last year, 60 Minutes produced an upbeat feature on Khan Academy’s efforts — host Anderson Cooper called Khanmigo’s potential “staggering,” but tempered the observation by adding, “It’s still very much a work in progress.” Other media accounts have challenged Khan’s predictions that AI will revolutionize education anytime soon, with a Wall Street Journal reporter a year ago observing that Khanmigo didn’t consistently know how to round answers or calculate square roots and “typically didn’t correct mistakes when asked to double-check solutions.”
Khan Academy has said improvements are ongoing, but that at least a few errors are likely to persist. The organization stresses that Khanmigo remains “imperfect” and “still evolving.”
Future Outlook
In March, DiCerbo will appear at South by Southwest EDU, alongside Curriculum Associate’s Kristen Huff and Akisha Osei Sarfo of the Council of the Great City School to discuss how AI can improve school assessments. The panel will be moderated by The 74’s Greg Toppo, who spoke recently with DiCerbo in a wide-ranging interview about Khanmigo, its critics, and the role of AI in education.
Latest Developments and Insights
You’ve been with Khan Academy now for almost five years, and it’s been an eventful time. You’ve spent a lot of that time creating and improving Khanmigo. What are the latest developments?
We have learned a lot in what is coming up on two years since Khanmigo launched. In terms of what students are doing, we definitely see some interesting things we didn’t necessarily expect. Students who are English language learners really like and use the supports in other languages. We probably shouldn’t have been surprised, but always need to be reminded that it’s important to just have instruction on how to use new technology and tools, and what that looks like. For students, how do you ask good questions? And for teachers, how do you integrate it? So both professional learning for teachers and supports for students have been important things that we’ve added over time.
The other thing is that we have found that Khanmigo as a tutor works best when it is paired with educational content we have already created. It is better integrated and has lower error rates when it’s using, and has reference to, the existing problems that were written and verified by people — and not just the problems, but the [step-by-step] hints and the answers that already exist in our system. When it can reference those, Khanmigo is better. And when students are just working on the practice that is part of Khan Academy generally, they are using Khanmigo as an assistant and as a help to get unstuck.
Handling Criticisms and Improving
When we talked last year, you used that word “unstuck.” You guys have come in for some criticism from critics like Ben Riley and Dan Meyer, who say Khanmigo gets math wrong, among other things. How have these criticisms influenced the product?
Dan has very good classroom experience and is extremely knowledgeable about teaching math. So many times, the things he says align to conversations internally that we’re already having. And the things he says are things that we end up changing and doing. We always appreciate criticism that helps us improve and move on. A lot of our work has been on things like working to better evaluate math accuracy, improve it, and get the balance right between how much Khanmigo gives help versus asks questions — all of the things we’re working to tune and get right in that sweet spot for what the student needs.
Impact of AI on Education
When I was writing the piece last year about IBM Watson and the effort to make it into a tutor, you expressed a cautious optimism that despite all the failures we’ve seen, this time was different. It’s been almost a year now. Have your feelings changed about AI tutoring generally and Khanmigo specifically?
I would still characterize how I feel as cautiously optimistic. I don’t think this is The Golden Ticket that’s going to save us all and be the sole reason that educational outcomes improve. I do think it still can be an important tool in the toolbox.
Future Directions and Partnerships
Does the change in presidential administrations have any bearing on your work, given the interest in AI and support from big tech companies like Open AI and Meta?
There is a lot of noise about what may or may not happen. We are basically sticking to “What are our technology partners doing, and what are we able to then partner with them to build?” And we will see what actually comes to fruition and deal with it if and when anything actually happens. We’re not counting on anything either way.
Adapting to Industry Changes
What are your thoughts on the recent developments in the AI industry, such as the earthquake that happened with the Chinese AI startup Deep Seek? How do you see these affecting the future of AI in education?
Not specifically the Deep Seek piece, but it’s just part of what we have thought is likely to be the future — it’s just a little bit sooner than perhaps we thought. The models themselves become a commodity. Even since we launched, the prices have come down so far that it’s significant. We’re able to offer what we do at significantly lower prices, and that’s just likely to continue. And it’s not going to be the models themselves that are the “moat” or the differentiator — it’s going to be what people build with them.
