Edge models — Small Language Models (SLMs) — are optimised for deployment on consumer devices: smartphones, laptops, edge servers, and IoT hardware. Examples include Microsoft's Phi series, Google's Gemma series, and Meta's small-parameter Llama variants, alongside platform-specific models like Apple's on-device models — each lab iterates these SLM families frequently, so check current generations rather than pinning to a specific release. Optimisation techniques include aggressive quantisation (INT4/INT8), model distillation (train small model to mimic large), and architecture optimisations (grouped query attention, smaller MLP ratios). Runtimes: llama.cpp, Ollama, Apple MLX, ONNX Runtime, ExecuTorch. Use cases: privacy-preserving inference, offline assistants, low-latency IoT, and embedded AI.
What it means in practice
Edge Models is not just vocabulary; it is a design handle. In LLM model selection, this term usually appears when engineers are designing, reviewing, or troubleshooting real production flows rather than only naming the concept. It helps teams choose the right model/runtime balance across quality, speed, memory, governance, and cost.
Why engineers care
- It gives teams a shared name for the behaviour, risk, or architecture choice being discussed.
- It helps separate the goal from the implementation detail, so you can compare alternatives instead of copying a tool pattern blindly.
- It creates a useful checklist for reviews: inputs, outputs, failure modes, ownership, cost, latency, and measurement.
Production watch-outs
Benchmarks are only a starting point. Validate with your prompts, data, latency budget, concurrency pattern, and safety requirements.
Related context
Useful neighbouring concepts: Quantisation, Inference, Vllm, Model Serving.