"Mediocre!": Math prof's ChatGPT o1 "Strawberry" review
OpenAI has recently introduced its latest ChatGPT model, which they describe as a significant advancement in AI capability, excelling at complex reasoning tasks. However, not everyone shares the same enthusiasm. Renowned mathematician, Terence Tao, also known as "the Mozart of math," recently reviewed the model after putting it through a series of challenging mathematical tasks.
A Mixed Review
Tao, a Professor of Mathematics at the University of California, Los Angeles, shared his experience with the new GPT model, which he named "Strawberry." While he acknowledged that the model was an improvement over its predecessors, he noted that it still struggled with the most advanced research mathematical tasks. To him, interacting with ChatGPT o1 felt like advising a mediocre graduate student, albeit one with the potential for growth.
Testing the Limits
Tao subjected the GPT 01 model to a series of complex mathematical problems, pushing its capabilities to the edge. Despite showing signs of improvement, such as correctly identifying Cramer's theorem in one instance, the model still fell short in generating key conceptual ideas independently.
The Future of AI in Mathematics
While acknowledging the current superiority of human mathematicians in certain areas, Tao speculated about the potential for AI tools to reach and surpass the level of a competent graduate student with further iterations and advancements. He envisioned specialized models integrated into development environments, offering valuable support in formalization projects and research tasks.
Despite the advantages of AI tools in certain aspects, Tao emphasized the unique qualities and potential for growth that human students possess. He highlighted the importance of a holistic evaluation of individuals, considering various dimensions of skills and attributes beyond technical proficiency.
