Getting started with Amazon Bedrock Custom Model Import with Sarvam and CodeLlama
Amazon Bedrock is a fully managed serverless service that offers a choice of high-performing FMs from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon. While this should serve the needs for most of the customer use cases, you also have the option of leveraging SageMaker Jumpstart, a Machine learning (ML) hub with foundation models, built-in algorithms, and prebuilt ML solutions that you can deploy with just a few clicks, which are then provisioned on servers in AWS.
Custom Model Import Feature
Apart from these two options, if you needed to use other Foundation Models (as a serverless/managed service similar to Bedrock), which are not yet available on Amazon Bedrock, and have been customized in other environments like Amazon SageMaker or available in a model repository like HuggingFace, now you can! With the general availability (GA) of Amazon Bedrock Custom Model Import a few days back, you can now import and use these customized models alongside existing foundation models (FMs) and by using the same API.
Proof of Concept
This blog is my quick proof of concept showcasing Amazon Bedrock Custom Model Import feature by importing a couple of Foundation Models from HuggingFace into Amazon S3, and then importing these models into Amazon Bedrock. I chose two models hosted on HuggingFace— sarvam-2b-v0.5, a small, yet powerful language model pre-trained from scratch on 2 trillion tokens and trained to be good at 10 Indic languages + English and CodeLlama-7b-Instruct-hf, a model designed for general code synthesis and understanding.
Steps to Get Started
There are some pre-requisites before you get started, please consult the documentation. Amazon Bedrock Custom Model Import supports a variety of popular model architectures, including Meta Llama, Mistral, Mixtral, Flan and more.
I used the following commands to download the model files and upload them to an Amazon S3 bucket for both Sarvam and CodeLlama.
The S3 bucket directory looks something similar to this:
Tip: The HuggingFace repo has a .git directory, I don’t think they are needed for Bedrock Model import. Just consult documentation on the required files for Custom Model Import. You can save a lot of time during download/upload if you chose to ignore this directory!
The next few steps are fairly easy, and Amazon Bedrock has made it very easy. Once the model is uploaded to an S3 bucket or via SageMaker, you can “Import the Model”, which kicks off a job, and it took around 10–13 minutes for me, and voila, the Model was available for inferencing.
Testing the Models
With Sarvam, I tested with a few prompts in Hinglish (a question in Hindi written in English), Kannada (summarizing a news article about the drubbing the Indian cricket team is receiving down under!!) and Hindi. I tried a few coding related tasks with CodeLlama — coding, debugging.
My prompts were in this format:
My first preference would be to use the wide range of Foundation Models that are available in Amazon Bedrock, while the Custom Model Import should be an option if I need to use a specialized FM which is not yet available on Bedrock. While this was a quick proof of concept, you should also consider the following aspects mentioned in the AWS blog before moving to production:
Custom Model Pricing
Please refer to https://aws.amazon.com/bedrock/pricing for the latest pricing.
