Start With a Hosted Model

If your data may leave your network and you can live with a provider's retirement dates, use a hosted model. It's more capable than anything you'll serve yourself, and somebody else runs it. Open weights earn their place when control matters more than convenience.

Every hosted model comes with a retirement date you do not control. Llama, Meta's open-weight family, is the best-known way out of that: you download the weights, run them where you like, fine-tune them on your own data, and nobody can deprecate them for you. The price is that serving, scaling and evaluating the model become your responsibility. The real decision is when that trade is worth making, and how to do it properly when it is.

Before you download the weights
  • Your data is not allowed to leave your network, so a hosted API is not an option
  • A provider retired the model your prompts were tuned for, and you do not want that again
  • You want to fine-tune, and the hosted fine-tuning options are shrinking
  • You ran a model locally with one command and now have to serve it to real users

The Family Today

ModelReleasedGood for
Llama 4 Maverick and ScoutApril 2025The newest open generation: multimodal, mixture-of-experts, very large context windows
Llama 3.3 70BDecember 2024The proven general model, with the widest tooling and fine-tune ecosystem
Llama 3.2 1B and 3BSeptember 2024Laptops, phones and edge devices
Llama 3.2 11B and 90B VisionSeptember 2024Image understanding at small and large sizes

One piece of context matters for planning. The Llama name has not had a major release since Llama 4, because Meta's newer work moved to the Muse family. Muse Spark launched closed-weight, but Meta returned to open weights in August 2026 with Muse Glimmer, a 30-billion-parameter Apache 2.0 model, and has said Muse Spark 1.2 weights will follow, so check whether they've landed. Llama is still the most widely supported open family, but it is no longer automatically the strongest one. Before you commit, evaluate it against the other open-weight options for your task, including Muse Glimmer, DeepSeek V4, Qwen, Mistral Large 3 and gpt-oss.

The Licence Is Not Open Source

Llama is released under Meta's community licence, not an OSI-approved licence. Commercial use is allowed, with conditions: an acceptable use policy, attribution requirements, and a clause that requires a separate agreement for companies above a very large monthly-active-user threshold. For most teams this is fine. It still needs to go through the same legal review as any other licence, and it is a real difference from the Apache 2.0 and MIT licences that some competing models use.

✅

Check it: Find out who read the licence and the acceptable use policy for the exact version you run. If nobody did, do that before the model reaches a product.

Serving: One Tool for Your Laptop, Another for Production

Ollama is the fastest way to try a model: one command downloads a quantised build and starts a local server. It's tuned for local use first, so benchmark it at your real concurrency before relying on it. For heavy production traffic I'd use an inference server built for concurrency. vLLM is the usual choice, because continuous batching lets one GPU serve many requests at once.

# Development
ollama run llama3.3

# Production: an OpenAI-compatible server with continuous batching
vllm serve meta-llama/Llama-3.3-70B-Instruct \
  --tensor-parallel-size 2 \
  --max-model-len 32768

Both expose an OpenAI-compatible endpoint, so application code does not need to know which one is behind it.

from openai import OpenAI

client = OpenAI(base_url="http://llm.internal:8000/v1", api_key="not-used")

reply = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct",
    messages=[{"role": "user", "content": "Summarise this incident report: ..."}],
    temperature=0.2,
    max_tokens=400,
)
✅

Check it: Load test your server with a realistic number of simultaneous users and realistic prompt lengths. Compare the throughput with the single-request figure.

Sizing: Memory Is the Constraint

GPU memory decides what you can run. Quantisation trades a small amount of quality for a large reduction in memory.

Llama 3.3 70BApproximate memoryNotes
16-bit (full precision)140 GBTwo 80 GB GPUs
8-bit70 GBClose to full quality
4-bit40 GBThe common production choice, with a small quality loss

Leave headroom above these figures. The context you hold in memory for each concurrent request takes space as well, and it grows with both context length and the number of simultaneous users. Size for peak concurrency, not for a single request.

✅

Check it: Add the memory for the weights at your quantisation level to the context held for your peak number of concurrent requests. Compare the total with the GPU memory you have.

When an API Is Still the Better Answer

Self-hosting used to win on cost at volume. That argument has weakened. Hosted economy models now cost around $0.10 per million input tokens, while two 80 GB GPUs rent for several thousand dollars a month whether they are busy or idle. You need very high and steady utilisation before owning the hardware is cheaper, and a self-hosted 70B model is not as capable as a current frontier model.

The reasons that still hold are these:

  • Data boundary. The data cannot leave your network, or must stay in a jurisdiction no provider serves.
  • Stability. You need a model that behaves the same in two years as it does today.
  • Customisation. You need fine-tuning that hosted providers no longer offer for their current models.
  • Latency and offline use. The model has to run next to the application or on the device.

There is a middle path. Hosted inference providers and the cloud model catalogues serve Llama behind an API, which gives you the open model without the operations, and leaves the door open to moving it in-house later.

⚠️ Common Mistakes

1. Using the wrong chat template

Each model expects its own special tokens around system, user and assistant turns. The wrong template does not raise an error. It quietly lowers quality. Use the tokenizer's built-in chat template instead of formatting prompts by hand.

2. Benchmarking with one request at a time

Throughput under concurrency is what production depends on, and it differs greatly from single-request speed. Load test with a realistic number of simultaneous users and realistic prompt lengths.

3. Treating open weights as open source

Read the licence and the acceptable use policy before the model reaches a product, and again when you move to a new version.

4. Skipping evaluation because the model is free

A model you host still needs a regression suite. Quantisation level, serving settings and template changes can all move quality, and only your own tests will show it.

Before You Commit

Meta's most recent model in October 2026 was Muse, and I found no new Llama release that month, so check the release notes before you plan around the family's future. Open-weight rivals such as Qwen 3.8 and DeepSeek V4 are also worth including in your eval.

Try It on Your Own System This Week

Three checks before you commit to serving Llama, about an hour in total:

  1. Write down which of the four lasting reasons applies to you. If the only reason is price, do the utilisation sum first.
  2. Run your eval on the quantised model you would really serve. Quantisation, serving settings and templates all move quality.
  3. Put one other open-weight family in the same eval. Llama is the most widely supported one. It is no longer automatically the strongest.

Key Takeaways

✅ Key Lessons
  • Choose Llama for control, not for price. Data boundary, stability and fine-tuning are the lasting reasons.
  • Ollama for development, vLLM for production. Both speak the OpenAI API, so switching is usually small, but test the tool and structured-output features you rely on.
  • Size GPU memory for concurrency, and expect to run 4-bit quantisation in production.
  • The open Llama line has slowed. Compare it with other open-weight families before committing.
  • The licence allows commercial use with conditions. Have it reviewed.