Anmol Sharma on LinkedIn: #LLM #Gemini #Google #OpenAI #GPT4 #Coding #CodeLlama #Llama #NLP #MachineLearning
Leading Teams bringing Generative AI and LLMs to Production
I just crossed 10,000+ lines of code in my personal project (a FastAPI/ReactJS webapp), and you know what? A good chunk (nearly 95% of it) was written with the help of AI. 🤯
Specifically, models like GPT-4, Code Llama, and (my current favorite) Gemini 1.5 Pro. If you're not using LLMs to write/modify/explain your code, you're making a massive mistake.
Context is Key
What's been fascinating is seeing how much context matters. Gemini 1.5 Pro with its 1 million token context window (2M window seems to be not available widely yet), can practically hold my entire codebase in its "memory." This means I don't have to feed it individual files anymore; I can just ask it questions as if it's a teammate who's been working on the project from day one. 🤝
Output Quality Matters
The difference in output quality is staggering. It writes code that adheres to my project's conventions and principles far better than models with smaller context windows. RAG-based approaches like GitHub Copilot feel clunky in comparison, and end up giving me sub-par results.
And since the release of GPT-4o, the original GPT-4 has massively downgraded in quality for me when generating reliable code. 😔
The Takeaway
Context is king. 👑 As AI models evolve, those with the ability to grasp larger contexts will undoubtedly reshape the landscape of software development.
To view or add a comment, sign in
A Watermark for Large Language Models
A new #paper from University of Maryland shows that potential harms of large language models (especially open source ones) can be somewhat mitigated by embedding invisible yet algorithmically detectable watermarks in the generated text.
Key Insights
- Efficient Watermarking: Embed signals in text with minimal impact on quality.
- Detection: Open-source algorithm to detect watermarks without model access.
- Robust and Secure: Supports both public and private watermarking modes for enhanced security.
Why This Works
The watermarking framework selects a randomized set of "green" tokens before generating a word, softly promoting their use during sampling. This method ensures the watermark is embedded with negligible impact on text quality. The watermark can be detected using a statistical test with interpretable p-values, even without access to the language model's API or parameters.
How It Works
A set of "green" tokens is chosen before each word is generated. These tokens are softly promoted during the sampling process. Finally, an open-source algorithm efficiently detects the watermark from a short span of tokens.
Practical Applications
- Finetuned Models: You can add watermarks to your finetuned models without significant overhead.
- Open Source Models: Watermarking can be applied to open-source models, enhancing their security and transparency.
This framework has been tested with multi-billion parameter models from the OpenPretrainedTransformer (OPT) family, demonstrating strong robustness and security against attacks.
