Two Voice Devs • A podcast on Spotify for Podcasters
Join Allen Firstenberg and Roger Kibbe as they delve into the exciting world of local, embedded LLMs. Despite encountering some technical gremlins along the way, our exploration continues. We uncover the reasons behind this shift, the potential benefits for consumers and vendors, and the challenges that developers will encounter in this evolving landscape.
We discuss the importance of "killer features" for driving adoption, the significance of fine-tuning and LoRA adapters, and the potential impact on autonomous agents and the emergence of an appless future.
Resources:
- developer.android.com/ai/aicore
- machinelearning.apple.com/research/introducing-apple-foundation-models
Key Discussion Points and Timestamps:
- 00:20 - Why are vendors embedding LLMs into operating systems?
- 04:40 - What are the benefits for consumers?
- 09:40 - Opportunities for app developers
- 14:10 - Power of LoRA adapters and fine-tuning for smaller models
- 17:40 - Comparing Apple, Microsoft, and Google's approaches
- 20:10 - Challenges of multiple LLM models in a single browser
- 23:40 - Handling browser compatibility with local LLMs
- 24:10 - The "three-tiered" system for local, cloud, and third-party LLMs
- 27:10 - Potential for an "appless" future driven by browsers and local AI
- 28:50 - Impact of local LLMs on autonomous agents
Don't miss out on this insightful conversation about the future of conversational AI development! It's time to embrace these transformative technologies and stay ahead of the curve.
