In this article
Edge models — Small Language Models (SLMs) — are optimised for deployment on consumer devices: smartphones, laptops, edge servers, and IoT hardware. Examples: Phi-4 Mini (3.8B), Gemma 3 (4B), Llama 3.2 (3B), Apple OpenELM. Optimisation techniques include aggressive quantisation (INT4/INT8), model distillation (train small model to mimic large), and architecture optimisations (grouped query attention, smaller MLP ratios). Runtimes: llama.cpp, Ollama, Apple MLX, ONNX Runtime, ExecuTorch. Use cases: privacy-preserving inference, offline assistants, low-latency IoT, and embedded AI.
What it means in practice
Edge Models is not just vocabulary; it is a design handle. In LLM model selection, this term usually appears when engineers are designing, reviewing, or troubleshooting real production flows rather than only naming the concept. It helps teams choose the right model/runtime balance across quality, speed, memory, governance, and cost.
Why engineers care
- It gives teams a shared name for the behaviour, risk, or architecture choice being discussed.
- It helps separate the goal from the implementation detail, so you can compare alternatives instead of copying a tool pattern blindly.
- It creates a useful checklist for reviews: inputs, outputs, failure modes, ownership, cost, latency, and measurement.
Production watch-outs
Benchmarks are only a starting point. Validate with your prompts, data, latency budget, concurrency pattern, and safety requirements.
Related context
Useful neighbouring concepts: Quantisation, Inference, Vllm, Model Serving.