Deploying and managing Llama 4 fashions includes a number of steps: navigating complicated infrastructure setup, managing GPU availability, guaranteeing scalability, and dealing with ongoing operational overhead. What should you might tackle these challenges and focus instantly on constructing your functions? It’s attainable with Vertex AI.
We’re thrilled to announce that Llama 4, the most recent era of Meta’s open massive language fashions, is now typically out there (GA) as a totally managed API endpoint in Vertex AI! Along with Llama 4, we’re additionally saying the final availability of the Llama 3.3 70B managed API in Vertex AI.
Llama 4 reaches new efficiency peaks in comparison with earlier Llama fashions, with multimodal capabilities and a extremely environment friendly Combination-of-Consultants (MoE) structure. Llama 4 Scout is extra highly effective than all earlier generations of Llama fashions whereas additionally delivering important effectivity for multimodal duties and is optimized to run in a single-GPU setting. Llama 4 Maverick is essentially the most clever mannequin choice Meta offers at present, designed for reasoning, complicated picture understanding, and demanding generative duties.
With Llama 4 as a totally managed API endpoint, now you can leverage Llama 4’s superior reasoning, coding, and instruction-following capabilities with the convenience, scalability, and reliability of Vertex AI to construct extra refined and impactful AI-powered functions.
This publish will information you thru getting began with Llama 4 as a Mannequin-as-a-Service (MaaS), spotlight the important thing advantages, present you the way easy it’s to make use of, and contact upon price concerns.
Uncover Llama 4 MaaS in Vertex AI Mannequin Backyard
Vertex AI Mannequin Backyard is your central hub for locating and deploying basis fashions on Google Cloud by way of managed APIs. It provides a curated number of Google’s personal fashions (like Gemini), open-source fashions, and third-party fashions — all accessible by means of simplified interfaces. The addition of Llama 4 (GA) as a managed service expands this choice, providing you extra flexibility.
Accessing Llama 4 as a Mannequin-as-a-Service (MaaS) on Vertex AI has the next benefits:
1: Zero infrastructure administration: Google Cloud handles the underlying infrastructure, GPU provisioning, software program dependencies, patching, and upkeep. You work together with a easy API endpoint.
2: Assured efficiency with provisioned throughput: Reserve devoted processing capability in your fashions at a set charge, guaranteeing excessive availability and prioritized processing in your requests, even when the system is overloaded.
3: Enterprise-grade safety and compliance: Profit from Google Cloud’s sturdy safety, information encryption, entry controls, and compliance certifications.
Getting began with Llama 4 MaaS
Getting began with Llama 4 MaaS on Vertex AI solely requires you to navigate to the Llama 4 mannequin card throughout the Vertex AI Mannequin Backyard and settle for the Llama Group License Settlement; you can not name the API with out finishing this step.
After you have accepted the Llama Group License Settlement within the Mannequin Backyard, discover the particular Llama 4 MaaS mannequin you want to use throughout the Vertex AI Mannequin Backyard (e.g., “Llama 4 17B Instruct MaaS”). Be aware of its distinctive Mannequin ID (like meta/llama-4-scout-17b-16e-instruct-maas), as you will want this ID when calling the API.
Then you may instantly name the Llama 4 MaaS endpoint utilizing the ChatCompletion API. There is not any separate “deploy” step required for the MaaS providing – Google Cloud manages the endpoint provisioning. Under is an instance of easy methods to use Llama 4 Scout utilizing the ChatCompletion API for Python.
import openai
from google.auth import default, transport
import os
# --- Configuration ---
PROJECT_ID = "<YOUR_PROJECT_ID>"
LOCATION = "us-east5"
MODEL_ID = "meta/llama-4-scout-17b-16e-instruct-maas"
# Receive Utility Default Credentials (ADC) token
credentials, _ = default()
auth_request = transport.requests.Request()
credentials.refresh(auth_request)
gcp_token = credentials.token
# Assemble the Vertex AI MaaS endpoint URL for OpenAI library
vertex_ai_endpoint_url = (
f"https://{LOCATION}-aiplatform.googleapis.com/v1beta1/"
f"initiatives/{PROJECT_ID}/areas/{LOCATION}/endpoints/openapi"
)
# Initialize the shopper to make use of ChatCompletion API pointing to Vertex AI MaaS
shopper = openai.OpenAI(
base_url=vertex_ai_endpoint_url,
api_key=gcp_token, # Use the GCP token because the API key
)
# Instance: Multimodal request (textual content + picture from Cloud Storage)
prompt_text = "Describe this landmark and its significance."
image_gcs_uri = "gs://cloud-samples-data/imaginative and prescient/landmark/eiffel_tower.jpg"
messages = [
{
"role": "user",
"content": [
{
"type": "image_url",
"image_url": {"url": image_gcs_uri},
},
{"type": "text", "text": prompt_text},
],
}
]
# Elective parameters (consult with mannequin card for specifics)
max_tokens_to_generate = 1024
request_temperature = 0.7
request_top_p = 1.0
# Name the ChatCompletion API
response = shopper.chat.completions.create(
mannequin=MODEL_ID, # Specify the Llama 4 MaaS mannequin ID
messages=messages,
max_tokens=max_tokens_to_generate,
temperature=request_temperature,
top_p=request_top_p,
# stream=False # Set to True for streaming responses
)
generated_text = response.selections[0].message.content material
print(generated_text)
# The picture incorporates...
Necessary: At all times seek the advice of the particular Llama 4 mannequin card in Vertex AI Mannequin Backyard. It incorporates essential details about:
- The precise enter/output schema anticipated by the mannequin.
- Supported parameters (like temperature, top_p, max_tokens) and their legitimate ranges.
- Any particular formatting necessities for prompts or multimodal inputs.
Value and quota concerns
Utilizing the Llama 4 as Mannequin-as-a-Service on Vertex AI operates on a predictable mannequin combining pay-as-you-go pricing with utilization quotas. Understanding each the pricing construction and your service quotas is crucial for scaling your software and managing prices successfully when utilizing the Llama 4 MaaS on Vertex AI.
With regard to pricing, you pay just for the prediction requests you make. The underlying infrastructure, scaling, and administration prices are included into the API utilization value. Discuss with the Vertex AI pricing web page for particulars.
To make sure service stability and honest utilization, your use of Llama 4 as Mannequin-as-service on Vertex AI is topic to quotas. These are limits on components such because the variety of requests per minute (RPM) your undertaking could make to the particular mannequin endpoint. Discuss with our quota documentation for extra particulars.
What’s subsequent
With Llama 4 now typically out there as a Mannequin-as-a-Service on Vertex AI, you may leverage one of the crucial superior open LLMs with out managing required infrastructure.
We’re excited to see what functions you’ll construct with Llama 4 on Vertex AI. Share your suggestions and experiences by means of our Google Cloud neighborhood discussion board.
