Google Leverages Anthropic's Claude to Enhance Gemini AI Performance
Google is utilizing Anthropic’s AI model, Claude, for performance benchmarking and evaluating its Gemini AI model’s outputs against those produced by Claude, according to a report by TechCrunch.
With a focus on criteria such as accuracy, truthfulness, and verbosity, this detailed evaluation process allows up to 30 minutes per prompt, ensuring a comprehensive comparison between the two AI models.
Internal communications have shown that Claude’s responses often prioritize safety more rigorously than Gemini’s. For example, Claude may refuse to respond to prompts it deems unsafe, such as those involving role-playing as a different AI assistant. In contrast, there have been instances where Gemini’s responses included inappropriate content.
.png)
Anthropic’s terms of service explicitly prohibit the use of Claude for developing or training competing AI models without prior approval. When questioned, Google did not confirm whether it had obtained such permission from Anthropic.
Shira McNamara, a spokesperson for Google DeepMind, mentioned that while comparing model outputs is standard industry practice, any implication that Anthropic’s models were used to train Gemini is inaccurate. This situation also raises questions about the ethical implications of using a competitor’s AI model for internal evaluations.
Furthermore, concerns have emerged regarding the expertise of contractors evaluating Gemini’s responses. Reports suggest that some contractors are required to assess outputs in domains beyond their professional knowledge, potentially leading to inaccuracies.
As AI technology continues to advance rapidly, the techniques employed by major tech companies to refine and improve their models are facing heightened scrutiny.
