DeepSeek's New AI Model: A Powerful Open Challenger - Betamind.ai
A Chinese lab has recently introduced one of the most powerful "open" AI models called DeepSeek V3. This AI model, developed by DeepSeek, has been made available under a permissive license, enabling developers to download and customize it for various applications, including commercial use.
Features of DeepSeek V3
DeepSeek V3 is specifically designed to handle a wide range of text-based tasks. According to DeepSeek, its internal benchmarks indicate that V3 surpasses both downloadable models and closed AI models that are only accessible via API.
In coding competitions on Codeforces, DeepSeek V3 has demonstrated superior performance over other models. It has also excelled on Aider Polyglot, showcasing its ability to generate code that integrates seamlessly with existing codebases.
DeepSeek reveals that V3 was trained using a massive dataset comprising 14.8 trillion tokens. To put this into perspective, 1 million tokens equate to approximately 750,000 words.
Scale and Efficiency
DeepSeek V3 stands out for its massive scale, boasting 671 billion parameters (685 billion on AI development platform Hugging Face). This significantly exceeds the parameters of other models, such as Llama 3.1 405B with 405 billion parameters.
Despite its impressive scale, DeepSeek's training methodology for V3 appears to be cost-effective. The model was trained using Nvidia H800 GPUs in a data center over a period of about two months, with the company investing around $5.5 million in its development. This cost is notably lower than that associated with models like GPT-4.
Regulatory Constraints
However, DeepSeek's model is subject to political constraints. For example, it avoids engaging with sensitive topics like Tiananmen Square. As a Chinese entity, DeepSeek is closely monitored by China's internet regulators to ensure that its models align with "core socialist values." This oversight often leads to the model refraining from discussing controversial subjects.
Future Developments
Recently, DeepSeek introduced DeepSeek-R1 as a response to OpenAI’s o1 reasoning model. With backing from High-Flyer Capital Management, the organization is focused on advancing superintelligent AI.
