Anthropic Claude 3.5 Sonnet Launched; Beats ChatGPT 4o | Beebom
Just after releasing the Claude 3 models three months back, Anthropic has now introduced a much improved Claude 3.5 Sonnet model. It’s not the largest model from Anthropic’s lab, yet it beats ChatGPT 4o and Gemini 1.5 Pro, at least in several benchmarks. Claude 3.5 Sonnet is a mid-tier model and it brings 2x faster speed than the largest Claude 3 Opus model.
API Price and Features
Anthropic has kept the API price the same for the Sonnet 3.5 model with a context window of 200K tokens. For general users, it’s available for free on claude.ai (visit) and supports both image and document uploads. Keep in mind that there is a rate limit for free users.
Benchmarks Comparison
Coming to benchmarks, Claude 3.5 Sonnet beats GPT-4o in nearly all benchmarks except MMLU and MATH, but the difference is very marginal. In HumanEval that tests coding abilities, Claude 3.5 Sonnet scores 92% whereas GPT-4o scores 90.2%. In GPQA Diamond which evaluates graduate-level reasoning, the new Sonnet model achieves a score of 59.4% whereas GPT-4o stands at 53.6%.
.png)
With 0-shot prompting in the MMLU test, Claude 3.5 Sonnet gets 88.3% and OpenAI’s GPT-4o model gets 88.7%. From the table, you can infer that Anthropic has developed a highly capable model that outranks both GPT-4o and Gemini 1.5 Pro models.
Vision Capabilities
Next, Claude 3.5 Sonnet is also a powerful vision model and again does better than GPT-4o in various visual reasoning tests. It’s very good at understanding and transcribing texts from illegible images. It’s also excellent at interpreting charts, graphs, and illustrations.
New Artifacts Tool
Moreover, Anthropic has announced a new Artifacts tool for Claude which works like OpenAI’s Code Interpreter tool. The Artifacts tool generates the code and creates AI-generated content in a separate interface. It’s not just limited to Python as it can work with other programming languages as well. For example, I created an SVG image of the Taj Mahal with the Artifacts tool on Claude Chat.
Future Models
Anthropic says Claude 3.5 Haiku and Claude 3.5 Opus are coming later this year. Overall, this new Sonnet model from Anthropic showcases impressive speed and intelligence, making it a potential replacement for ChatGPT 4o in everyday tasks.
