News
EveryLLM

Small and on-device

Phi, Gemma, Gemini Nano, Apple's on-device models and other compact models. 49 models on EveryLLM so far.

Small language models trade breadth for speed and cost. With from a few hundred million to around ten billion parameters, they can run on a laptop, a phone or a single modest graphics card, keep data on the device and answer without a network connection. Careful data selection, distillation from larger models and quantisation, storing each parameter in fewer bits, let today's small models match what much larger models did a few years earlier.

How small models got good

Microsoft's Phi series argued that data quality matters as much as size. Phi-1 (June 2023, 1.3 billion parameters) was trained largely on textbook-style material, and Phi-3-mini (April 2024) packed 3.8 billion parameters into a model that runs on a phone. Google's Gemma (February 2024), Meta's Llama 3.2 1B and 3B (September 2024) and Hugging Face's SmolLM followed. Many are distilled: trained to imitate the outputs of a larger teacher model.

On the device

Google put Gemini Nano on the Pixel 8 Pro in December 2023. Apple Intelligence, announced in June 2024, runs an on-device model of about three billion parameters and sends harder requests to Apple's own servers. Running locally cuts delay and keeps personal data on the phone, but it is limited by memory: a model stored at 4 bits per parameter needs about half a gigabyte for every billion parameters.

Famous small models

Still to be written: Xiaomi MiMo.

Further reading

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.