ChatGPT's Daily Operational Cost is ~USD 700,000 – Report
A chief analyst at a semiconductor research firm has recently reported that ChatGPT could cost OpenAI over USD 700,000 in operational expense every day. According to SemiAnalysis’s Dylan Patel, ChatGPT's massive operational cost is due to the huge amount of infrastructure it requires to calculate responses to hundreds of millions of prompts. Most of the cost is allocated to servers that the AI model requires to achieve the necessary computing power amidst the growing demand in AI. It is worth noting that this estimate is based on GPT-3, and the cost could be even higher with the GPT-4 model.
In response to this challenge, Microsoft is developing a chip called Athena to decrease the operational costs of running generative AI models. The project was initiated in 2019 after Microsoft’s one-billion-dollar deal with OpenAI, which required OpenAI to exclusively run its AI models on Microsoft’s Azure cloud servers. One reason behind the move is that Microsoft was trailing behind Google and Amazon in building its own in-house chips and wanted cheaper alternatives to Nvidia’s graphics processing units, which their models currently use.
According to reports, the said AI chip could be released for internal use by Microsoft and OpenAI this year, indicating that it could also benefit businesses that require a large amount of generative AI models for their daily operations.
