Llama: Open-Weight Models for Production

The Model You Are Allowed to Keep

Every hosted model comes with a retirement date you do not control. Llama, Meta's open-weight family, is the best-known way out of that: you download the weights, run them where you like, fine-tune them on your own data, and nobody can deprecate them for you. The price is that serving, scaling and evaluating the model become your responsibility. This article is about deciding when that trade is worth making, and how to do it properly when it is.

The Family Today

One piece of context matters for planning. Meta's newer flagship models are closed-weight, so the open Llama line has not had a major release since Llama 4. Llama is still the most widely supported open family, but it is no longer automatically the strongest one. Before you commit, evaluate it against the other open-weight options for your task, including DeepSeek V4, Qwen, Mistral Large 3 and gpt-oss.

The Licence Is Not Open Source

Llama is released under Meta's community licence, not an OSI-approved licence. Commercial use is allowed, with conditions: an acceptable use policy, attribution requirements, and a clause that requires a separate agreement for companies above a very large monthly-active-user threshold. For most teams this is fine. It still needs to go through the same legal review as any other licence, and it is a real difference from the Apache 2.0 and MIT licences that some competing models use.

Serving: One Tool for Your Laptop, Another for Production

Ollama is the fastest way to try a model: one command downloads a quantised build and starts a local server. It handles one request at a time well and many requests poorly. For production traffic use an inference server built for concurrency. vLLM is the usual choice, because continuous batching lets one GPU serve many requests at once.

# Development
ollama run llama3.3

# Production: an OpenAI-compatible server with continuous batching
vllm serve meta-llama/Llama-3.3-70B-Instruct \
  --tensor-parallel-size 2 \
  --max-model-len 32768

Both expose an OpenAI-compatible endpoint, so application code does not need to know which one is behind it.

from openai import OpenAI

client = OpenAI(base_url="http://llm.internal:8000/v1", api_key="not-used")

reply = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Summarise this incident report: ..."}],
    temperature=0.2,
    max_tokens=400,
)

Sizing: Memory Is the Constraint

GPU memory decides what you can run. Quantisation trades a small amount of quality for a large reduction in memory.

Leave headroom above these figures. The context you hold in memory for each concurrent request takes space as well, and it grows with both context length and the number of simultaneous users. Size for peak concurrency, not for a single request.

When an API Is Still the Better Answer

Self-hosting used to win on cost at volume. That argument has weakened. Hosted economy models now cost around $0.10 per million input tokens, while two 80 GB GPUs rent for several thousand dollars a month whether they are busy or idle. You need very high and steady utilisation before owning the hardware is cheaper, and a self-hosted 70B model is not as capable as a current frontier model.

The reasons that still hold are these:

• Data boundary. The data cannot leave your network, or must stay in a jurisdiction no provider serves.
• Stability. You need a model that behaves the same in two years as it does today.
• Customisation. You need fine-tuning that hosted providers no longer offer for their current models.
• Latency and offline use. The model has to run next to the application or on the device.

There is a middle path. Hosted inference providers and the cloud model catalogues serve Llama behind an API, which gives you the open model without the operations, and leaves the door open to moving it in-house later.

Key Takeaways