GloVe
Unsupervised word embeddings from global word co-occurrence counts
GloVe (Global Vectors) is an unsupervised learning algorithm that turns words into vectors, so that distance in the vector space reflects meaning similarity. It was built at Stanford University and released in 2014 as an open-source project, and it was meant as a rival to word2vec.
- Released in 2014 as open source from Stanford University
- Trains on global word-word co-occurrence counts from a corpus
- Combines matrix factorisation with local context window methods
- Recommended vector is the sum of the two learned vectors per word
- By 2022 seen as outdated next to transformer models like BERT
What it is
GloVe, short for Global Vectors, is a way of giving each word a list of numbers, called a vector, so that words close in meaning sit close together in that space. Stanford University released it in 2014 as open source. It learns from counts of which words appear near which other words across a whole corpus, rather than from any labels. The result shows clear linear patterns in the vector space. It was built to compete with word2vec, and its paper claimed several gains over that approach.
How it came about
The work came out of Stanford University and was published as an open-source project in 2014. Its authors were Jeffrey Pennington, Richard Socher and Christopher D. Manning. The design goal was to beat word2vec, and the original paper set out a number of improvements over it. By 2022 both methods had been overtaken by transformer models such as BERT, which stack attention layers on top of an embedding stage much like word2vec's.
Inside the model
For every word the method learns two vectors plus two bias terms. Training uses counts of how often word j sits in the window around word i, written Xij. The target is the log of the co-occurrence probability, and the model fits a dot product of the two vectors plus biases to that value. A weighting function f(Xij) ramps the loss up slowly with count, so rare pairs do not dominate; the paper found xmax = 100 and alpha = 3/4 worked well. At the end the two vectors are added to form the word's representation.
What it can do
GloVe captures relations such as synonyms, company-to-product links, and pairings like zip codes with cities. Its vector space shows linear substructure. A known limit is homographs: words spelled the same but with different senses get one vector, because the algorithm assigns a single set of vectors per word form. It is used by the spaCy library to build semantic features, matched with cosine similarity or Euclidean distance, and it served as the word representation layer in systems that detect psychological distress from patient interviews.
How to use it
GloVe was released as an open-source project. Trained models give four parameters per word, of which only the two vectors matter; the recommended final vector is their sum.
Reception and debate
The original paper presented GloVe as an improvement on word2vec and reported several advantages. By 2022 it was regarded as outdated, with transformer models such as BERT seen as the leading approach in natural language processing.
Why it matters
GloVe became a standard word representation method and was built into tools such as spaCy. It sits in the line of embedding work that includes word2vec and fastText, and it fed into later systems such as ELMo and BERT, which add neural attention layers on top of a similar embedding stage.
Questions people ask about GloVe
What does GloVe actually produce?
It produces a vector, a short list of numbers, for each word in the vocabulary. Words with similar meanings end up near each other in that space, and the distance between vectors tracks semantic similarity.
How is it trained?
It counts how often each pair of words appears within a context window across a corpus, then fits vectors so their dot product matches the log of those co-occurrence probabilities. A weighting function keeps rare pairs from distorting the fit.
How does it differ from word2vec?
GloVe uses statistics gathered over the whole corpus, while word2vec works from local context windows. The GloVe paper presented it as a competitor to word2vec and reported several improvements over it.
Is it still used today?
By 2022 both GloVe and word2vec were considered outdated. Transformer models such as BERT, which place attention layers over an embedding stage, are regarded as the state of the art in natural language processing.
What are its weaknesses?
It cannot handle homographs well. Words with the same spelling but different meanings share one vector, since the algorithm gives a single set of vectors to each word form.
Good reading and sources
- Official websiteOfficial site
- On the validity of pre-trained transformers for natural language processing in the software engineering domainarXiv, Article
The facts on this page come from the Wikipedia article "GloVe", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryLLM rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.