Search: pre training
WorldGPT-1
OpenAI's first GPT model, a 117M-parameter decoder-only transformer from 2018.Research · Open weights · 2018
WorldMM1
A family of multimodal language models up to 30B parameters, dense and MoE.Multimodal · Research only · 2024
🇺🇸 United StatesBART
A denoising encoder-decoder transformer for generation and comprehension tasksGeneral · Conditional licence · 2019
🇺🇸 United StatesDeBERTa
A BERT-style encoder with disentangled attention and an enhanced mask decoder.Encoder · Open weights · 2020
WorldELECTRA
A pre-training method that trains text encoders to spot replaced tokens.Research · Research only · 2020