ChatGPT's success could have come sooner, says former Google AI...
A co-author of Attention Is All You Need reflects on ChatGPT's surprise and Google's conservatism. In 2017, eight machine-learning researchers at Google released a groundbreaking research paper called Attention Is All You Need, which introduced the Transformer AI architecture that underpins almost all of today's high-profile generative AI models. 
The Transformer has made a key component of the modern AI boom possible by translating (or transforming, if you will) input chunks of data called "tokens" into another desired form of output using a neural network. Variations of the Transformer architecture power language models like GPT-4o (and ChatGPT), audio synthesis models that run Google's NotebookLM and OpenAI's Advanced Voice Mode, video synthesis models like Sora, and image synthesis models like Midjourney.
Insights into Transformer Development
At TED AI 2024 in October, one of those eight researchers, Jakob Uszkoreit, spoke with Ars Technica about the development of transformers, Google's early work on large language models, and his new venture in biological computing. 
Uszkoreit's Reflections
In the interview, Uszkoreit revealed that while his team at Google had high hopes for the technology's potential, they didn't quite anticipate its pivotal role in products like ChatGPT.
"There was a notion of, well, 'that could have happened.' I think it was less of a, 'Oh dang, they got it first' or anything of the like. It was more of a 'Whoa, that could have happened sooner.' Was I still amazed by just how quickly people got super creative using that stuff? Yes, that was just breathtaking," he shared.
Uszkoreit also discussed his departure from Google and the founding of Inceptive, emphasizing the importance of pursuing endeavors with a direct impact on people's lives.
Anticipating Innovation
Reflecting on the emergence of ChatGPT, Uszkoreit highlighted the importance of taking risks in technological advancements. He referenced past experiences with Google Translate to illustrate the evolution of transformative technologies. 
Regarding ChatGPT's reception, Uszkoreit acknowledged the unexpected speed and creativity with which users embraced the technology, underscoring the significance of user experimentation and feedback in driving innovation.
Exploring Biological Software
Post-Google, Uszkoreit co-founded Inceptive to apply deep learning to biochemistry, focusing on designing better medicines through "biological software." This innovative approach involves AI compilers translating specified behaviors into RNA sequences for impactful applications in biological systems. 
Uszkoreit's work exemplifies the intersection of advanced AI technologies with biomedical research, offering promising avenues for future medical breakthroughs.
