GPT models have been making a buzz in the AI industry due to their potential to create new content, including audio, text, code, images, videos, and simulations. While several believe in their immense potential, others are sceptical, given the models' ability to reproduce the work of renowned artists and writers in seconds and potentially replace several routine jobs. In this article, we list five trends that may change the course of Generative AI models.
Open-source language models
One trend is the growing popularity of open-source language models (LLMs). These models are faster, more private, and more flexible, and they are capable of doing things at a much lower cost than traditional models. Developers are flocking to open-source LLMs, such as Meta's Large Language Model (LLaMA), because they require less computing power and resources to test new approaches, validate others' work, and explore new use cases. Additionally, smaller models trained on more tokens are easier to fine-tune for specific product use cases, making them a more attractive option for developers.
The Rise of Low-Rank Adaptation
Another trend is the emergence of Low-Rank Adaptation of Large Language Models (LoRA), which has reduced the number of trainable parameters required for specific tasks. This has lowered the storage requirements for LLMs and enabled efficient task-switching during deployment without inference latency. LoRA also outperforms several other adaptation methods, including adapter, prefix-tuning, and fine-tuning. Developers can use LoRA to fine-tune LLaMA, further making open-source LLMs an attractive option.
Release of Dolly 2.0 by Databricks Inc.
Databricks Inc. released Dolly, an LLM that it trained for less than $30 to exhibit human-like interactivity, in March. A month later, Databricks Inc. released Dolly 2.0, a 12B parameter model based on the EleutherAI Pythia model family and fine-tuned exclusively on a new, high-quality human-generated instruction following dataset. The company has open-sourced Dolly 2.0 in its entirety, including the training code, dataset, and model weights, for commercial use, enabling any organization to create, own, and customize powerful LLMs without paying for API access or sharing data with third parties.
The Emergence of BLOOM
Hugging Face's BigScience Large Open-science Open-access Multilingual Language Model (BLOOM) is capable of generating text in 46 natural languages and 13 programming languages. Researchers can download, run, and study BLOOM to investigate the performance and behavior of recently developed LLMs.
The Limitations of Large Language Models
While LLMs may have several advantages, there are limitations to their capabilities. LLMs like GPT-3 can outperform humans at some tasks, but they may not comprehend what they read or write as we humans do. Additionally, these models use human supervisors to make them more sensible and less toxic. A new paper by Stanford University researchers suggests that the whole may not be greater than the sum of its parts when it comes to scaling AI models. Therefore, it is essential to hold our horses and gather more data from the opaque LLMs of OpenAI and Google.
Conclusion
Generative AI models have tremendous potential for creating new content and revolutionizing various industries. One of the key trends that may shape their future is the growing popularity of open-source LLMs, which are faster, more private, and more flexible than traditional models. Additionally, other developments such as LoRA, Dolly 2.0, and BLOOM are also making LLMs more accessible and customizable. However, it's essential to keep in mind the limitations of LLMs and gather more data to make informed decisions about their potential.
