OpenAI's ChatGPT costs up to $700,000 per day to operate, claims report
OpenAI's conversational AI bot ChatGPT has become a viral sensation since its inception but keeping it up and running is costing the company a fortune, according to a report by SemiAnalysis. It is estimated that OpenAI is spending as much as $694,444 per day to run the system, including around 3,617 HGX A100 servers with approximately 28,936 GPUs. The cost per query is estimated to be around 0.36 cents. The exorbitant cost is largely due to the power-hungry specialized chips necessary to operate the system.
Microsoft, one of the significant shareholders in OpenAI, is working on a new AI chip called 'Athena' that could replace the NVIDIA GPUs and significantly reduce ChatGPT's running costs.
The recently launched GPT-4 version for paying subscribers offers more accurate information and better protection against off-the-rails comments, but the cost of running it could be even higher than GPT-3. The authors of the study suggest that the current costs to run the software could be more than estimated.
The report further reveals that ChatGPT can generate functional code from scratch, raising fears that it could replace programmers eventually. However, a recent study by computer scientists suggests that code generated by the chatbot may not be secure.
The AI chatbot generates code falling well below minimal security standards applicable in most contexts. When asked whether the code it generated was secure, ChatGPT admitted that it was not. The researchers asked ChatGPT to generate 21 programs and scripts using C, C++, Python, and Java languages. The AI chatbot managed to write only five secure programs on the first try and came up with seven more secured code snippets after some prompting from the researchers. The results suggest that using ChatGPT to code apps could be dangerous in the foreseeable future.
Overall, while ChatGPT has gained a lot of popularity as an AI chatbot, it is becoming increasingly expensive to maintain and has some limitations when it comes to coding. Microsoft's Athena could eventually replace NVIDIA GPUs and reduce running costs significantly, but it remains to be seen if ChatGPT's coding limitations can be overcome.
