News
IndiaIndiaAI Mission Shortlists 12 Teams for Indigenous AI Models: Minister Updates Rajya SabhaChinaSarvam Prices Its Own Model Below DeepSeek In New Token FactoryIndiaSarvam AI faces backlash over giving access to Chinese modelsIndiaSarvam defends serving foreign AI models on Token Factory, says it runs on privately procured GPUsIndiaSarvam AI’s LLM researcher catches ChatGPT using his laptop without authorisation — it apologisesIndiaSarvam sells DeepSeek access while pursuing sovereign AI modelsIndiaSarvam faces backlash for offering Chinese AI models on its platformIndiaSarvam defends token factory strategy amid criticism over Chinese AI modelsReleasesAn updated Clairic and Clairic-flash are now open-weight, from the new Workers AI Superintelligence teamResearchOpenAI pens AI safety agreement with four firmsIndiaIndiaAI Mission Shortlists 12 Teams for Indigenous AI Models: Minister Updates Rajya SabhaChinaSarvam Prices Its Own Model Below DeepSeek In New Token FactoryIndiaSarvam AI faces backlash over giving access to Chinese modelsIndiaSarvam defends serving foreign AI models on Token Factory, says it runs on privately procured GPUsIndiaSarvam AI’s LLM researcher catches ChatGPT using his laptop without authorisation — it apologisesIndiaSarvam sells DeepSeek access while pursuing sovereign AI modelsIndiaSarvam faces backlash for offering Chinese AI models on its platformIndiaSarvam defends token factory strategy amid criticism over Chinese AI modelsReleasesAn updated Clairic and Clairic-flash are now open-weight, from the new Workers AI Superintelligence teamResearchOpenAI pens AI safety agreement with four firms
EveryLLM

State-space and recurrent

Mamba, RWKV and the models that avoid attention's cost on long texts. 12 models on EveryLLM so far.

State-space and recurrent models process text as a running summary rather than attending back over every earlier token. They carry a fixed-size state from one token to the next, so memory and time grow only in step with length, and generating each new token costs the same however long the text. The trade-off is recall: a fixed state can forget details that attention would look up exactly. Mamba and RWKV are the leading designs.

From S4 to Mamba

Structured state-space models, starting with S4 from Stanford in 2021, borrowed ideas from control theory to model long sequences efficiently. Mamba, published in December 2023 by Albert Gu of Carnegie Mellon and Tri Dao of Princeton, made the state depend on the input, letting the model choose what to keep, and matched transformers of similar size on language tasks. Mamba-2 followed in May 2024.

Recurrent revivals

RWKV, an open project led by Bo Peng and described in a 2023 paper, trains like a transformer but runs like a recurrent neural network. Other efforts include Microsoft's RetNet (2023) and xLSTM from Sepp Hochreiter's group in Linz (2024). In practice the largest releases mix these layers with some attention rather than dropping it: Nvidia's Nemotron-H models and IBM's Granite 4.0 (2025) use mostly Mamba-2 layers.

Famous state-space models

Further reading

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.