Bots Battle for Supremacy in Legal Reasoning – Part Five: Reigning ...
The champion of legal reasoning was determined to be Orion ChatGPT-4.5 in March 2025. This was reported in Part Three of the Battle of the Law Bots series. It defeated prior champ, Omni ChatGPT-4o. It had been AI legal reasoning champion of the world for a month after defeating several other Google Gemini and OpenAI models. This was reported in Parts One and Two of the series. Then 4.5 Orion was released in March and defeated Omni in reasoning tests. 
The next month, April 2024, ChatGPT-o3 was released. I pitted o3 against 4o expecting Omni to win. Surprisingly, little o3 defeated Omni in a convincing manner. This was reported in Part Four. This win qualified ChatGPT-o3, which I nicknamed Scorpio, to challenge Orion. That is what we do here in Part Five of the AI legal reasoning bot battles.
Orion vs. Scorpio
OpenAI in late April 2025 described the current champ, Orion, as follows: GPT-4.5 is OpenAI’s latest and most advanced language model, introduced as a research preview. It emphasizes enhanced pattern recognition, creative insight generation, and emotional intelligence, aiming to deliver more natural and reliable responses.. . . GPT-4.5 is available as a research preview to Plus, Pro, and Team users. OpenAI described GPT-o3, Scorpio, in this manner: GPT-o3 is designed to handle complex reasoning tasks with enhanced capabilities. . . . o3 excels in step-by-step logical reasoning, making it adept at solving intricate problems in mathematics, science, and programming.
Apparently OpenAI had not tested it in legal reasoning. They do not have legal experts on their teams. If they had, they would know, as I found out in Part Four, that its step by step reasoning abilities also makes it adept at solving intricate problems in law. But will it be good enough to beat Orion in legal reasoning?
According to legend, Orion’s ruthless hunting and bragging angered the Goddess Gaia. She knew there was one animal on her Earth he could not defeat, the scorpion. Gaia sent a little scorpio to attack Orion, which it did, stinging him dead. This in turn angered Artemis, aka Diana, the goddess of the hunt. She responded by promoting Orion into an immortal constellation. Not to be outdone, Gaia promoted the scorpion into a constellation, Scorpius, but they could never appear in the night sky at the same time.
Impressive Performances
After writing Part Four I learned that many other professional reviewers where also very impressed with o3. One AI reviewer who covers all models, Igor Pogany, mentioned how many experts now consider o3 to have attained AGI level. Pogany also mentioned that o3 is currently ranked by llm-stats.com as the top general-purpose AI in the world.
Although I admittedly favor little Scorpio, I have gone out of my way to keep this a fair fight as I will explain next. First of all, to make sure neither had any inside information and this was a closed book exam, I picked two Bar Exam essay questions from July 2024 Californian Bar Exam.
As an extra precaution, I gave them both the test at the same time to prevent internal instructions carryover or second taker advantage. I only provided them with the model answers later when I asked for critiques. Again, I made very sure neither model had seen any answers prior to the test.
