Small language models (1-7B parameters) run on consumer hardware, phones, and edge devices. Models like Phi-3, Gemma 2, and TinyLlama offer surprising quality for their size. Key techniques: distillation, quantization (GGUF Q4), ONNX runtime, and WebLLM for browser inference.
The 5-Mode Loop
5 of 5 modes available
Read · See · Animate · Test · Build