Sebastian Raschka, PhD on LinkedIn: Building LLMs from the Ground Up
If you’re interested in delving into Large Language Models (LLMs) and understanding their inner workings, I have prepared a 3-hour coding workshop presentation that covers implementing, training, and utilizing LLMs.
Below is a breakdown of the workshop content to give you an overview of what the video entails. The video includes clickable chapter marks, allowing you to navigate directly to topics of interest:
Workshop Overview
Introduction to LLMs
Understanding LLM input data
Coding an LLM architecture
Pretraining
Loading pretrained weights
Instruction finetuning
Benchmark evaluation
Evaluating conversational performance
Conclusion
This workshop provides a hands-on approach to learning about LLMs, offering a comprehensive understanding of the topic.
New LLM Pre-training and Post-training Paradigms
Exploring the latest advancements in Large Language Models (LLMs), we delve into the pre-training and post-training pipelines of four cutting-edge models:
- Alibaba’s Qwen 2
- Apple Intelligence Foundation Language Models
- Google’s Gemma 2
- Meta AI’s Llama 3.1
These models represent the forefront of LLM development, showcasing innovative techniques and approaches in the field. By examining their pre-training and post-training processes, we gain valuable insights into effective LLM practices.
Community Engagement and Collaborative Learning
I am thrilled to see the enthusiasm and engagement from readers organizing study groups and deriving substantial value from my book on building Large Language Models from scratch.
The community of LLM Trailblazers has been actively applying their learnings, dedicating an average of 3 hours per week to enhance their skills.
For those keen on delving into the world of LLMs, the next batch of study groups will commence in September. Join the LLM Trailblazers community to deepen your understanding of this trending topic without the distractions.
Additionally, current LLM Trailblazers are embarking on Builder Weeks, engaging in various projects such as fine-tuning LLMs, challenging BPE with MorphPiece, and rebuilding Llama 3.1. The collaborative spirit and diverse projects within the community foster a conducive learning environment where members support each other's endeavors.
Spread the word and encourage more individuals to join the LLM Trailblazers community, where knowledge sharing and skill development thrive.
