How to deploy Llama 3.1 in the Cloud: A Comprehensive Guide
Llama 3.1, the latest series of open-weight LLMs released by Meta AI under a community license, represents a complex AI language model designed to generate and understand human alike text. It’s part of the LLaMA (Large Language Model Meta AI) series which aims to provide powerful tools for natural language processing tasks. The Llama 3.1 models, available in three sizes: 8B, 70B, and 405 B, demonstrate exceptional performance that surpasses other open-weight models of similar sizes. In this blog, we will focus on the 70 B model, with the 405B model to be covered in our next Llama series blog. The Llama 3.1 research report confirms that the 405B model matches the benchmark performance of GPT-4, further highlighting their superior performance.
NodeShift Cloud Setup
For the purpose of this tutorial, we will use a GPU-powered Virtual Machine offered by NodeShift; however, you can replicate the same steps with any other cloud provider of your choice. Visit the NodeShift Cloud website (https://app.nodeshift.com/) and create an account. Once you've signed up, log into your account. Follow the account setup process and provide the necessary details and information.
NodeShift GPUs offer flexible and scalable on-demand resources like NodeShift Virtual Machines (VMs) equipped with diverse GPUs ranging from H100s to A100s. These GPU-powered VMs provide enhanced environmental control, allowing configuration adjustments for GPUs, CPUs, RAM, and Storage based on specific requirements.
GPU Deployment Process
Navigate to the menu on the left side. Select the GPU VMs option, create a GPU VM in the Dashboard, click the Create GPU VM button, and create your first deployment. In the "GPU VMs" tab, select a GPU Model and Storage according to your needs and the geographical region where you want to launch your model.
For the purpose of this tutorial, we are using the RTX 4090 Model to deploy Llama 3.1 70B. After this, select the amount of storage to run meta-llama/meta-lama-3.1-70 B. You will need at least 135 GB of storage.
There are two authentication methods available: Password and SSH Key. SSH keys are a more secure option, in order create them, head over to our official documentation: (https://docs.nodeshift.com/gpus/create-gpu-deployment)
Virtual Machine Setup
Next, you will need to choose an image for your VM. We will be deploying Llama 3.1 70b on an NVIDIA Cuda Virtual Machine, it’s a proprietary and closed source parallel computing platform that will allow you to install Llama 3.1 on your GPU VM.
After choosing the image, click the ‘Create’ button, and your VM will be deployed.
You will get visual confirmation that your machine is up and running. NodeShift GPUs can be connected to and controlled through a terminal using the SSH key provided during GPU creation.
