LLM Economics: ChatGPT vs Open-Source
Large Language Models (LLMs) have become increasingly popular in recent years, thanks to models like BERT, GPT, and BART. These models are capable of performing multiple language tasks like sentiment analysis, Q&A, classification, and more. However, these models are so resource-intensive that they were economically challenging to deploy at scale, which changed with the arrival of ChatGPT. In this article, we will explore the economic viability of using ChatGPT versus open-source LLMs deployed in AWS, and which model is better for specific use cases.
ChatGPT
The ChatGPT API is priced by usage, costing $0.002/1K tokens. For lower usage in the 1000's of requests per day range, ChatGPT works out cheaper than using open-sourced LLMs deployed to AWS. Let's say you process 1000 small chunks of text per day, each chunk being a page of text — so 500 words or 667 tokens. This comes to $0.002/1000x667*1000= ~$1.3 a day, which is not too bad. But if you are processing a million such documents a day, then it is $1,300 per day or ~0.5 Million$ per year, making ChatGPT a major expense in a multi-million dollar business.
Open-Source LLMs
While open-source models are free to use, the infrastructure to host and deploy them is not. Earlier transformer models like BERT could easily be run and fine-tuned on personal computers with a good CPU and basic GPUs, but LLMs are more resource-intensive. A common solution is to use cloud providers like AWS to host and deploy such models. However, AWS costs for hosting open-source models can be expensive. For example, hosting an open-source LLM like Flan-UL2 on AWS is $150 for 1000 requests a day, and $160 for 1 M requests a day.
Which Model is Better?
The responses from ChatGPT and GPT-4 are more relevant than those from open-source LLMs. However, open-source models are catching up quickly. Companies want to fine-tune open-source models on their specific data sources. ChatGPT and subsequent OpenAI models might not perform as well as open sourced models fine-tuned on domain-specific data. Already, we are seeing domain-specific models like BloombergGPT making powerful moves in Generative AI.
Ultimately, which model is better depends on the use case. For smaller language models like BERT that are 100's of millions of parameters, you can get away with using cheaper instances like ml.m5.xlarge that is $0.23/hour and ~5$ a day. These models are also pretty powerful and are more task and training data-specific compared to LLMs that seem to understand the complex nuances of language. Keep in mind that deployment costs may reduce drastically as demands increase, and smaller language models like BERT are still a great option for specific tasks.
In conclusion, both ChatGPT and open-source LLMs have their advantages and disadvantages. It ultimately depends on the specific use case and the level of usage required.
