News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
Talkie

Talkie

A 13B small language model trained only on pre-1931 public domain English text

Talkie is an experimental small language model released in April 2026 by Nick Levine, David Duvenaud and Alec Radford. It is trained only on English texts published before 1931, so its training data is out of copyright in the United States, and it is used to study forecasting, cultural change and copyright questions in AI.

Share on WhatsApp

  • Trained only on English texts published before 1931
  • 260 billion tokens from books, newspapers, journals, patents and case law
  • Two 13B checkpoints: a base model and a chat version
  • Training compute supplied by Anthropic
  • Released under the Apache License 2.0

What it is

Talkie, styled in lower case, is an experimental small language model released in April 2026 by Nick Levine, David Duvenaud and Alec Radford. It is billed as a "vintage language model", meaning it was trained only on historical text. The family has two 13 billion parameter checkpoints: talkie-1930-13b-base, the base model, and talkie-1930-13b-it, a post-trained version built to run a chat interface. Its training corpus stops at the end of 1930, so everything it learned from is in the public domain in the United States. The project exists partly to cut the legal risk of publishing training data, and partly as a research instrument for studying prediction, culture and model behaviour.

How it came about

The idea began as an attempt to train a model on historical data and then test how well it could forecast later events, including how far ahead it could predict. The authors also wanted to study cultural change and how a model sees itself, and to probe whether a language model could reach inventions or scientific findings on its own. Their prompt for this came from a thought experiment by Demis Hassabis, who asked whether a model trained only up to 1911 could rediscover general relativity as Albert Einstein did. The term "vintage language model" is credited to Owain Evans. Talkie followed other vintage efforts such as Ranke 4B, Mr Chatterbox and Machina Mirabilis, and drew on Calcifer Computing's work on Temporal Language Models.

Inside the model

Talkie was trained on 260 billion tokens of English text published before 1931, drawn from the Institutional Data Initiative, the Common Pile and the Internet Archive. The 31 December 1930 cutoff follows United States copyright rules, where works published from 1923 to 1977 are protected for 95 years. The corpus covers books, newspapers, periodicals, scientific journals, patents and case law. Because old scans are messy, the team tried several optical character recognition methods and built a vintage OCR system of their own. Anthropic supplied the training compute. For the chat version, a separate post-training pipeline fine-tuned the base model on historical structured texts such as etiquette and letter-writing manuals, cookbooks, dictionaries and encyclopedias.

What it can do

The model can be used to simulate past language use and to examine behaviour free of modern content. Reviewers found it weaker than current models on standard benchmarks such as HumanEval, and the research team accepts these limits. Contamination is the main weakness: bad metadata and editorial notes leak post-cutoff material into the corpus, so talkie knows about Franklin Delano Roosevelt and Adolf Hitler. The chat checkpoint adds further anachronisms, because it was fine-tuned with synthetic conversations produced by a Claude Opus model. Novelist Robin Sloan tested whether it could find Claude Shannon's link between electric circuits and Boolean logic; talkie denied any such correspondence exists.

How to use it

Talkie is released under the Apache License 2.0. Both checkpoints, talkie-1930-13b-base and talkie-1930-13b-it, are published, and there is a demo chat interface anyone can try. The project has an official website and repositories on GitHub and HuggingFace.

Safety and testing

A large part of the safety argument is legal rather than behavioural: training only on pre-1931 public domain works was intended to reduce liability around releasing model data. The team is open about the model's shortcomings, including its anachronistic knowledge. Simon Willison calls the base model "vegan", meaning it was trained entirely on licensed or out-of-copyright material, but says the chat model does not qualify because proprietary synthetic data from Claude Opus entered during fine-tuning.

Reception and debate

Outlets used the public demo to probe the model's worldview; Decrypt's Jose Antonio Lanz wrote about its take on Adolf Hitler. The Decoder and The Register both reported that it trails modern models on benchmarks like HumanEval. The Harvard Observer stressed how much such projects depend on libraries and archives, naming Harvard University's Institutional Data Initiative, and on the quality of both inputs and outputs. Writing in The Observer, John Naughton placed talkie at the centre of the AI copyright debate, saying it is by design what plaintiffs in the piracy lawsuits argue the big labs should have built. Robin Sloan called the project "a triumph".

Why it matters

Talkie is treated as a reference case in the argument over training data and copyright, because its corpus is entirely out of copyright and its data can be released. It sits alongside other vintage language models and extends Calcifer Computing's work on Temporal Language Models. Commentators point to it as evidence of what libraries and archives make possible, and as a testbed for questions about forecasting, cultural change and whether models can reach scientific insights unaided.

Questions people ask about Talkie

What makes Talkie a "vintage" language model?

The label, credited to Owain Evans, covers models trained only on historical text. Talkie's corpus stops at 31 December 1930, so it simulates past language use and avoids modern contamination. Other examples include Ranke 4B, Mr Chatterbox and Machina Mirabilis.

Why does the training data end in 1930?

The cutoff follows United States copyright law, where works published between 1923 and 1977 are protected for 95 years. Anything from before 1931 is in the public domain there, which lowers the legal risk of publishing the training data alongside the model.

How big is Talkie and how do I get it?

There are two 13 billion parameter checkpoints: talkie-1930-13b-base and talkie-1930-13b-it, the post-trained chat version. Both are released under the Apache License 2.0, with a demo chat interface and repositories on GitHub and HuggingFace.

Does it really know nothing after 1930?

No. Wrong metadata and editorial notes bring later material into the corpus, so the model is aware of figures such as Franklin Delano Roosevelt and Adolf Hitler. The chat version adds more anachronisms because it was fine-tuned with synthetic chats from a Claude Opus model.

Why does Talkie matter beyond research?

Writers such as John Naughton see it as a concrete answer to the copyright debate: it is built the way plaintiffs in the piracy lawsuits say large labs should have built their models. It also shows what libraries and archives contribute to model training.

Good reading and sources

Where this page comes from

The facts on this page come from the Wikipedia article "Talkie (language model)", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.

It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

The weekly digest

Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.