John Schulman, co-founder of the software company OpenAI, is the mastermind behind ChatGPT - a chatbot that is based on company's generative pre-trained language models. Schulman has become a global sensation for his remarkable work on developing ChatGPT.
Journey of John Schulman with AI
During a recent interview, Schulman shared his journey with AI and how he became interested in Neuroscience and Robotics. After completing his undergraduate degree in physics from the California Institute of Technology, Schulman initially came to UC Berkeley to do a Ph.D. in Neuroscience. However, he found himself more interested in AI and eventually switched to machine learning and robotics.
At Berkeley, Schulman did his lab rotation with Pieter Abbeel, and this is where he got excited about research on helicopter control and towel-folding robots. Schulman then asked to switch to the electrical engineering and computer sciences (EECS) department, where he became one of the pioneers of deep reinforcement learning, which combines deep learning with reinforcement learning.
In 2015, Schulman co-founded OpenAI, where he led the reinforcement learning team that developed ChatGPT. Schulman was interested in OpenAI's mission, which was ambitious and already thinking about artificial general intelligence (AGI). AGI refers to AI that can match or exceed human abilities in almost every area. Schulman defines AGI as AI that is beyond human ability in many ways, like GPT-4, which he states is getting really general.
The Innovations behind ChatGPT
Schulman's initial work on a new technique called 'reinforcement learning with human feedback' (RLHF) is one of the main innovations behind ChatGPT. Schulman used RLHF to help direct how the AI behaves by rating how it responds to different inquiries. Schulman's idea to apply RLHF to ChatGPT was inspired by a paper called "Deep reinforcement learning from human preferences" from OpenAI. The OpenAI safety team worked on this paper because they wanted to align models with human preferences - try to get models to do what humans want.
After the success of RLHF in other areas, Schulman saw the potential in the research direction of using language models for summarization. He joined OpenAI's effort and worked on ChatGPT. Schulman saw the gradual improvement of the models and was not surprised by how well it worked after the initial interactions.
John Schulman's journey with AI and the innovations behind ChatGPT has changed the AI industry. Schulman's future work in AGI will be exciting to watch.
