News
ReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior ReleaseReleasesOVHcloud unveils Olala mini, its first in-house LLMChinaA 9B model beats a 1T model on FinQA, trained with Crusoe Managed RLReleasesThe internal metaphysics of an LLMReleasesMapping Language Model Internal Representations to Physical Laws and Scientific ReasoningResearchCan China keep rogue AI in check? Meet the start-up testing the top modelsFrontierMicrosoft aims to challenge OpenAI, Anthropic with cheaper, specialized AI modelsOpen modelsMicrosoft rolls out new first-party models and updates to agentic AI continuous improvementFrontierMicrosoft unveils AI models for coding, cybersecurity as it takes on Anthropic, OpenAIFrontierGoogle is making it easier to opt out of Gemini AI model training – here’s howFrontierOpenAI Unveils New Astra AI Model After Scrapping Prior Release
EveryLLM
AlexaTM 20B

AlexaTM 20B

A 20-billion-parameter multilingual sequence-to-sequence model from Amazon's Alexa team.

AlexaTM 20B is a multilingual sequence-to-sequence language model with 20 billion parameters, trained on a mix of denoising and causal language modelling tasks. It was introduced in August 2022 by Amazon Alexa AI researchers, who reported strong few-shot results across summarisation, translation and multilingual understanding benchmarks.

Share on WhatsApp

  • 20 billion parameters, multilingual sequence-to-sequence design
  • Pre-trained on denoising plus causal language modelling tasks
  • One-shot summarisation results reported above PaLM 540B
  • One-shot translation reported strong for lower-resource languages
  • Zero-shot results reported above GPT-3 175B on SuperGLUE and SQuADv2

What it is

AlexaTM 20B is a large multilingual language model built on a sequence-to-sequence design, meaning it has separate parts for reading an input and producing an output rather than only predicting the next word. The team behind it works on Amazon's Alexa. Announced in August 2022, it carries 20 billion parameters, the adjustable numerical weights that store what a model learns. Its purpose is to handle many languages and many kinds of language task with little or no task-specific training, using few-shot prompting instead. The researchers present it as an alternative to decoder-only designs of the kind used by GPT-3 and PaLM.

How it came about

The model was described in an arXiv paper first posted on 2 August 2022, with a revised version the following day. Sixteen researchers from Amazon Alexa AI are credited on the work. Their aim was to test whether a sequence-to-sequence model pre-trained on a blend of denoising and causal language modelling could learn new tasks from a handful of examples more effectively than decoder-only systems. The paper reports comparisons against PaLM 540B and GPT-3 175B.

Inside the model

The system is a multilingual sequence-to-sequence model with 20 billion parameters. Pre-training combined two objectives: denoising, where the model must recover corrupted text, and causal language modelling, where it predicts what comes next. That mixture is central to the authors' argument, since they say it makes the model a more efficient few-shot learner than a decoder-only model trained only to predict forward. The paper does not publish layer counts, tokenizer details, training token totals or hardware figures in the abstract.

What it can do

The authors report that the model reached state-of-the-art results on one-shot summarisation, beating the far larger 540B PaLM decoder. They also claim top results on one-shot machine translation, particularly for lower-resource languages, across nearly all the language pairs it supports: Arabic, English, French, German, Hindi, Italian, Japanese, Marathi, Portuguese, Spanish, Tamil and Telugu, measured on Flores-101. In zero-shot use they say it surpasses GPT-3 175B on SuperGLUE and SQuADv2, and leads on XNLI, XCOPA, Paws-X and XWinograd.

How to use it

The work was published as an arXiv paper. The abstract does not describe a public release of weights, an API or a licence.

Why it matters

The authors argue their findings make a strong case for sequence-to-sequence training as a workable substitute for decoder-only approaches at large scale, and for building few-shot systems that way.

Questions people ask about AlexaTM 20B

What kind of model is AlexaTM 20B?

It is a multilingual sequence-to-sequence language model with 20 billion parameters. Unlike a decoder-only model that just predicts the next word, it reads an input and generates an output, and it was trained on both denoising and causal language modelling tasks.

Who built it and when?

Amazon Alexa AI researchers presented it in an arXiv paper posted on 2 August 2022, with a revision the next day. Sixteen authors are listed on the work.

Which languages does it cover?

The paper names Arabic, English, French, German, Hindi, Italian, Japanese, Marathi, Portuguese, Spanish, Tamil and Telugu among the pairs it supports, and reports translation results on the Flores-101 dataset.

How does it compare with larger models?

The researchers say it beat the 540B PaLM decoder on one-shot summarisation and outperformed GPT-3 175B in zero-shot tests on SuperGLUE and SQuADv2, despite being far smaller than either.

Can I download or use it?

The abstract describes a research paper only. It does not mention released weights, an API, a product integration or a licence for the model.

Good reading and sources

    Where this page comes from

    There is no English Wikipedia article on AlexaTM 20B yet, so the facts on this page come from the sources listed above: arxiv.org/abs/2208.01448. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryLLM wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.

    It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.

    The weekly digest

    Every Monday: the model of the day, new pages on EveryLLM, and the conferences and anniversaries of the week ahead.