An embedding is a list of numbers — typically a few hundred to a few thousand — that represents a piece of text (or an image, audio, or code). Two texts that mean similar things have embeddings that are close together by some distance measure. Two texts that mean different things have embeddings that are far apart.

That is the whole idea. Embeddings turn meaning into geometry, and geometry has fast algorithms.

What "close" means

The standard distance is cosine similarity. Two embeddings are similar when the angle between their vectors is small. The score ranges from -1 to 1; in practice, anything above 0.7 is usually considered a strong match for natural text.

Euclidean distance also works. For most retrieval tasks, cosine is the default because it is invariant to the magnitude of the vectors — only their direction matters.

Where embeddings are used

  • Semantic search. Embed a query, find the closest embeddings in a pre-computed index, return the source documents. The basis of agentic RAG.
  • Retrieval-augmented generation. The agent embeds the user's question, retrieves the most relevant chunks, and feeds them into the model as context. Covered in agentic RAG: retrieval that thinks.
  • Deduplication. Embed a stream of documents; near-duplicate embeddings flag the same content in different words.
  • Clustering. Group documents by embedding proximity to organize large corpora and surface themes.
  • Graph memory. In graph-based memory, edges between nodes are often weighted by embedding similarity.

How it differs from keyword search

Keyword search matches exact words. "How do I refactor a Python class?" returns documents containing "refactor," "Python," and "class." Semantic search, using embeddings, returns documents about refactoring Python classes — including ones that never use those exact words.

The two approaches are complements, not substitutes. Most production retrieval uses both: keyword search for exact matches (names, IDs, error messages) and semantic search for paraphrases.

Dimensions, models, and cost

Embedding models produce vectors of fixed dimensionality. Common sizes in 2026: small models at ~384 dims (fast, cheap, lower quality); ~768 dims as a mid-size default; ~1024 to 1536 dims for larger models with better quality; and ~3072 dims at the top-of-line, where returns diminish for most tasks.

Storage grows linearly with dimensions and document count. A million 1024-dim vectors is roughly 4 GB of float32 storage, plus index overhead. Comparison cost grows with dimensions too, though a good approximate nearest-neighbor index (HNSW, IVF) keeps query time near-constant.

One practical pitfall

Out-of-distribution text. An embedding model knows what it was trained on. Embed text that is too far from that distribution — narrow-domain jargon, a language with little training data, code in an unusual style — and the geometry stops being meaningful. "Close" no longer implies "similar."

A few cases that bite:

  • Language mismatch. Many models are English-heavy. Embedding English and Japanese with the same model produces vectors that are not comparable across languages.
  • Code vs prose. Some models embed code well; others do not. For source code, pick a model trained on code.
  • Short queries vs long documents. A single word and a paragraph live in different parts of the space. Asymmetric retrieval — a different embedding strategy for query vs document — handles this. Naive symmetric retrieval does not.
  • Stale embeddings. Models update. Re-embed your store with a new model and you have to redo it all; old and new vectors are not in the same space.

The takeaway: embeddings are a tool, not a magic wand. They make semantic search possible at scale, and they are the substrate of RAG, dedup, and clustering — but they have failure modes. Test your retrieval quality on real queries before you trust the system in production.

For the practical loop that uses embeddings, see the tool entry and the broader concept piece on memory.