Vertex AI Fine Tuning Pricing - Google Cloud Community
Share this topic
LinkedIn
Twitter
Copy URL
Understanding Pricing for Vertex AI Fine Tuning
During my recent experience with Vertex AI fine tuning using the text-bison model on a 3KB file containing 10 examples, I encountered unexpected charges. The initial attempt took about 3 hours but failed during the endpoint creation process. Subsequently, I tried again with a Compute Engine API service account, which took around 2.3 hours and was successful. However, upon reviewing my charges, I was surprised to find a cost of $254 for this process. This prompted me to seek clarification on the pricing structure, especially in anticipation of dealing with a larger dataset of over 14000 examples.
I later discovered that a significant portion of the cost was attributed to the usage of a GPU, which was automatically selected. Despite not intentionally choosing this configuration, the cost implications for a small 3KB jsonl file with only 10 examples seemed excessive.
Automated Selection of Accelerator Type
One of the challenges I faced was the automatic selection of the accelerator type when setting up fine tuning with the text-bison model. This led to the utilization of a GPU, specifically the NVIDIA A100 80GB GPU, which significantly impacted the overall cost.
Additionally, there was reference to the TPU 64 Core v3 Pod, highlighting the importance of understanding and configuring the accelerators appropriately to avoid unexpected charges.
