Embeddings: what cosine similarity measures, and where it misleads

About this article: AI-assisted writing, reviewed against the cited sources. This is source research, not a lab validation of the procedures. Check the stated versions and your own recovery requirements before making changes.

Cosine similarity answers one question: how aligned are the directions of two vectors? Retrieval systems ask that question of embedding vectors and then treat the answer as "these texts are about the same thing". The number is real. The interpretation is an extra step, and it fails in ways you can see in the formula and in measured geometry of language-model vectors.

An AI writing system produced this article for NetLeverage's editorial studio. Saying two texts are "close" here means a high cosine between vectors. It is not a claim that a model experiences similarity, and it is not a self-report.

Nothing below was run. The library formula is from the scikit-learn 1.9.1 documentation as served on 10 November 2026. The geometry findings are from the two papers cited, on the models they measured, not from a private index.

Why it matters

A retrieval stack embeds a query and a pile of passages, then sorts passages by cosine with the query. The top of that list is what the generator sees. If cosine is high for a reason other than shared meaning, the generator is handed the wrong passage with a confident score. If cosine ignores a signal you needed, such as how long or how peaked a vector is, two passages can tie when an operator expected one of them to win.

The formula is general. Ethayarajh studied BERT, ELMo and GPT-2; Timkey and van Schijndel studied BERT, RoBERTa, GPT-2 and XLNet. Both papers examine token representations by layer, not pooled vectors from a retrieval service. Applying their findings to retrieval is an inference to test on the actual embedding model.

What the number is

scikit-learn 1.9.1 documents cosine similarity as the normalised dot product. For vectors $X$ and $Y$,

The numerator is the sum of coordinate-wise products. Each denominator is the Euclidean length. The same page says that on L2-normalised data this function matches a linear kernel, which is the plain dot product. That is the whole trick. Cosine throws length away, then takes the dot product of what remains.

Write a vector as length times a unit vector, $X=\Vert X\Vert \widehat{X}$. Substitute, and the lengths cancel, provided neither length is zero:

where $\theta$ is the angle between them. The result lies between −1 and 1 when the coordinates are real. 1 means the same direction. 0 means a right angle. −1 means opposite directions. The formula does not guarantee a positive score for an embedding model.

Four consequences follow before any paper about language models.

Length is not similarity. A vector and another vector twice as long in the same direction score 1. If a model uses length to encode confidence, frequency or how much of the passage was informative, cosine will not see it. Two unit vectors are compared, whatever the originals were.

The zero vector is undefined. If either norm is 0, the denominator is 0. A padding vector, a failed embed call that returned zeros, or a row you accidentally cleared does not have a cosine. Libraries choose conventions for this undefined angle. The cited scikit-learn example returns 0 for a zero row. That can look like a genuine orthogonal result, so explicitly check for zero vectors instead of treating every returned zero as “unrelated”.

Dot-product search is cosine search only after normalisation. Because cosine is the dot product of the unit vectors, an index that returns the largest inner product returns the same order as cosine when every stored vector and the query have been L2-normalised. If the stored vectors still have different lengths, the inner product ranks length as well as angle. A system can be "using cosine" in the application code and "using inner product" in the index, and the top hit will differ. The equality scikit-learn states — cosine equals the linear kernel on L2-normalised data — is the check. Normalise both sides, or do not pretend the two rankings match.

One large coordinate can own the sum. The dot product is a sum. A single coordinate where both vectors are large contributes a large term. Coordinates where both are near zero contribute almost nothing. Cosine does not know which coordinates were meant to carry meaning.

What the papers measured

Ethayarajh, in "How Contextual are Contextualized Word Representations?" (arXiv:1909.00512v1, 2 September 2019), measured BERT, ELMo and GPT-2. The paper reports anisotropy in non-input layers; ELMo's input layer is an exception. Isotropic would mean directions spread evenly. Anisotropic means they occupy a narrow cone. He reports that in GPT-2's last layer, two random words have, on average, almost perfect cosine similarity. The same paper defines self-similarity of a word in a layer as the average cosine between its contextual vectors across different contexts. Same-word pairs still score higher than different-word pairs, but self-similarity drops in upper layers. He also reports that, on average, after adjusting for anisotropy, the first-principal-component estimate places an upper bound below 5% on average for variance explained by a static vector. The adjustment and upper-bound interpretation matter; this is not the raw variance of every layer.

That combination is the trap. In the GPT-2 last-layer example, if random words already sit at a cosine near 1, a score of 0.95 is not evidence they match. Ethayarajh spells out the baseline problem: if any two words average 0.99, then a self-similarity of 0.95 is low, not high. A retrieval threshold copied from a tutorial ("keep anything above 0.8") is meaningless until you know the cosine of random pairs in that model and that layer.

Timkey and van Schijndel, "All Bark and No Bite" (arXiv:2109.04404, EMNLP 2021), ask why those cosines look like that. They find that a few rogue dimensions, often just one to three, dominate cosine similarity and Euclidean distance in the contextual models they tested. Those dimensions sit far from the origin and have high variance. The dimensions that dominate the similarity score are not the dimensions that matter to the model's behaviour, on their measurements. There is a mismatch: the geometry cosine sees is not the geometry the rest of the network is using.

Their correction is standardisation: use a representative corpus to estimate each coordinate's mean and standard deviation, then subtract that mean and divide by that standard deviation, then compute similarity in that rescaled space. They report that cosine after that step tracks human similarity judgements more closely, and that the apparent collapse of representational quality in later layers was largely the rogue coordinates. Standardisation is a post-process on the vectors you compare. It is not a claim that the language model changed its mind. It also does not, by itself, fix a retrieval corpus you embedded last month and an index you built on the raw vectors. If you standardise queries and not the stored passages, you have changed the space on one side only.

Read both papers as measurements on named models, not as a law of every embedding API sold since. A model trained to produce already-normalised, less anisotropic vectors can look different. You still owe yourself the random-pair baseline on the model you deploy.

Where operators get misled

A high score against everything. If the cone is narrow, every passage scores high against every query. The ranking can still be useful if the differences are stable, but the absolute threshold is not. Sort, then look at the gap between rank 1 and rank 10. A pile of 0.91, 0.90, 0.90 may reflect a high baseline; those three scores alone neither prove a cone nor establish relevance.

A low score that is actually a match. Upper layers, in Ethayarajh's measurements, pull same-word vectors apart because they are more context-specific. A cosine that fell from the layer below can mean "this occurrence is doing a different job", which is what you wanted from a contextual model, not a retrieval failure. Compare within one layer. Do not mix layer 2 of one model with layer 12 of another and call it a similarity.

Magnitude you meant to keep. Deduplicating near-copies is a fair use of cosine, because direction is the question. Ranking "how much does this passage answer the query" may not be, if the embedding's length carried that. Cosine cannot tell those products apart once both vectors are unit length.

A few coordinates masquerading as a topic. Timkey and van Schijndel's rogue dimensions are this failure with a measured cause. Two passages can share a large value on one coordinate that is associated with token position or punctuation, and cosine will reward it. Standardising, or at least inspecting per-coordinate variance before you trust a score, is the check they argue for. Dropping the coordinates that contribute most to cosine and re-scoring a sample of known matches against known non-matches tells you whether your index was a topic index or a rogue-coordinate index.

The zero and the near-zero. A vector of zeros has no direction. Clipping a dominant coordinate changes direction; removing all nonzero coordinates creates a zero vector with no defined angle. Guards belong on the embedding call: reject non-finite values, reject norms below a threshold you choose, and do not write those rows into the index.

Inconsistent normalisation. If the corpus was L2-normalised at index time and the query is not, a cosine function that normalises both at score time is still cosine. With unit stored vectors, a nonzero unnormalised query scales every inner-product score by the same positive query norm. The ranking for that query remains the cosine ranking, but scores and absolute thresholds do not equal cosine. Normalising both sides makes both rankings and values agree.

What to choose and why

Our recommendation, not a library default: store the embedding model, the layer or pooling you used, and whether vectors were L2-normalised or standardised. A cosine without that record cannot be compared with last month's cosine.

The safe order when retrieval looks randomly confident

1. Write down the formula you actually compute. If the index is inner product, confirm both sides are L2-normalised. Unit stored vectors suffice for the ranking of one nonzero query; normalise both sides when the numerical scores or cross-query thresholds must equal cosine.

2. Score random pairs. Embed passages that should not match and record the mean and the spread of cosines, in the same layer you retrieve from. A threshold has to sit above that cloud, not above a number from a blog.

3. Look at coordinate variance. If one or a few coordinates have a variance far above the rest, you are in the situation Timkey and van Schijndel describe. Standardise a copy of the vectors and see whether known matches rise and known non-matches fall. If they do, rebuild. Do not standardise only the query.

4. Reject bad vectors before they enter the index. A zero vector has no angle, and near-zero norms can be numerically unstable. A library may silently return 0, which resembles “unrelated”; catch these data faults explicitly.

5. Judge rank gaps, not a single absolute score. In an anisotropic space the absolute cosine is a property of the cone. The gap between the hit you hoped for and the hits you did not is the evidence.

Risks and limits

• The formula is general. The cone is not. scikit-learn states the normalised dot product for whatever matrix you pass. Ethayarajh's "almost perfect" random-word cosine is a GPT-2 last-layer measurement, not a constant for every embedding model.
• Less than 5% variance explained by a static vector is the anisotropy-adjusted, first-principal-component upper-bound result across the reported models and layers. It is an argument for contextual vectors, not a quality score for your index.
• Rogue dimensions, often one to three, are what Timkey and van Schijndel found on the contextual models they tested. Standardisation helped those measurements. It is not a guaranteed lift on a retrieval set you have not labelled.
• Cosine ignores length by construction. If you needed length, cosine did not fail. The choice of cosine did.
• Zero norms are outside the definition. The 1.9.1 page does not state a policy in words, but its example returns 0 for a zero row.
• No embedding product, hosted API or NetLeverage search service is described or measured.

Sources

1. scikit-learn 1.9.1, sklearn.metrics.pairwise.cosine_similarity — https://scikit-learn.org/stable/modules/generated/sklearn.metrics.pairwise.cosine_similarity.html (opened 10 November 2026). $K(X,Y)=⟨X,Y⟩/(\Vert X\Vert \Vert Y\Vert )$; equivalent to linear_kernel on L2-normalised data.
2. K. Ethayarajh, How Contextual are Contextualized Word Representations?, arXiv:1909.00512v1, 2 September 2019 — https://arxiv.org/pdf/1909.00512 (opened 10 November 2026). Anisotropy; self-similarity; GPT-2 last-layer random pairs; less than 5% of variance explained by a static embedding.
3. W. Timkey and M. van Schijndel, All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality, arXiv:2109.04404, EMNLP 2021 — https://arxiv.org/pdf/2109.04404 (opened 10 November 2026). One to three rogue dimensions can dominate cosine; standardisation as the correction they measured.

This article was originally published on 10 November 2026.