Small and on-device
Phi, Gemma, Gemini Nano, Apple's on-device models and other compact models. 49 models on EveryLLM so far.
Small language models trade breadth for speed and cost. With from a few hundred million to around ten billion parameters, they can run on a laptop, a phone or a single modest graphics card, keep data on the device and answer without a network connection. Careful data selection, distillation from larger models and quantisation, storing each parameter in fewer bits, let today's small models match what much larger models did a few years earlier.
How small models got good
Microsoft's Phi series argued that data quality matters as much as size. Phi-1 (June 2023, 1.3 billion parameters) was trained largely on textbook-style material, and Phi-3-mini (April 2024) packed 3.8 billion parameters into a model that runs on a phone. Google's Gemma (February 2024), Meta's Llama 3.2 1B and 3B (September 2024) and Hugging Face's SmolLM followed. Many are distilled: trained to imitate the outputs of a larger teacher model.
On the device
Google put Gemini Nano on the Pixel 8 Pro in December 2023. Apple Intelligence, announced in June 2024, runs an on-device model of about three billion parameters and sends harder requests to Apple's own servers. Running locally cuts delay and keeps personal data on the phone, but it is limited by memory: a model stored at 4 bits per parameter needs about half a gigabyte for every billion parameters.
Famous small models
Still to be written: Xiaomi MiMo.
Further reading
Nothing matches those filters yet. Take one off, or check back: new pages are written every day.