Embeddings
Vectors of floating point numbers whose distance measures relatedness, the representation underneath semantic search and retrieval.
Karl-Gustav Kallasmaa, Founder & CEOLast updated An embedding is a numerical representation of a piece of content, arranged so that similar meanings end up near each other. OpenAI's embeddings documentation defines it in one sentence: an embedding is a vector, that is a list of floating point numbers, and the distance between two vectors measures their relatedness. Everything else about them follows from that.
The size of the vector is a model choice. OpenAI documents text-embedding-3-large as returning 3,072 dimensions by default and text-embedding-3-small as returning 1,536, with both supporting a reduced size through a dimensions parameter. Those numbers describe the representation, not the amount of text: one paragraph and one page each become a single vector of the same length.
What they are for
The documented uses are broader than search alone: OpenAI lists search, where results are ranked by relevance to a query string; clustering, where text strings are grouped by similarity; recommendations, where items with related text are suggested; anomaly detection; diversity measurement; and classification, where a string is assigned to its most similar label.
For anyone thinking about AI search, the first is the one that carries the weight. An assistant that answers from retrieved documents needs a way to find candidate passages quickly, and comparing query vectors to passage vectors is that way. Embeddings are therefore the layer beneath semantic search and one of the standard components of retrieval-augmented generation.
How they differ from tokens, and from a vector database
Three things sit close together and are frequently swapped:
- [Tokens](/glossary/tokens) are the units of text a model reads and writes. They are counted, billed and limited.
- An embedding is one vector describing one chunk of text. It has a dimension count, which is not a token count, and no meaning as a sequence.
- A vector database is the store and index that makes searching millions of those vectors fast. It is infrastructure, not representation, and swapping it changes nothing about what the numbers mean.
There is also a constraint that is easy to miss and expensive to discover: each embedding model defines its own space. Distances are comparable only within one model's output, so upgrading the model means re-embedding the entire corpus. That migration cost, rather than the storage cost, is what usually determines how often a production system changes model.
Where they fail
Vector similarity is very good at paraphrase and poor at exact tokens. A query and a passage that share no words can be neighbours, which is precisely the point. But an exact identifier — a part number, an error code, a SKU, a legal citation — has no useful neighbourhood in meaning space, and similarity search will happily return something that looks like it and is not it.
This is why serious retrieval systems are hybrid, running lexical matching alongside vector search and combining the results. It is also why chunking decisions matter so much: an embedding of a 3,000-word page is an average of everything on it, and averages of mixed material are close to nothing in particular. A page split into sections with their own headings produces vectors that mean something.
How to act on it
- Give each section one job. A section that answers one question embeds cleanly; a section covering four embeds into the middle of nowhere.
- Write the qualifier into the sentence. Retrieval returns a span without the surrounding page, so a claim that depends on a caveat two paragraphs up arrives without it.
- Keep exact identifiers exact and in the text. Do not rely on similarity to recover a part number; state it where a lexical match can find it.
- Use headings that state the claim rather than teasing it. Heading text is often embedded with the passage and does real work.
- Do not confuse the dimension count with quality. A larger vector is not automatically a better retrieval result; chunking and query construction usually matter more.
Frequently asked questions
What is an embedding?
A vector of floating point numbers whose distance from another vector measures relatedness.
How many dimensions?
Model-specific: 3,072 for text-embedding-3-large and 1,536 for text-embedding-3-small by default.
Can I mix embeddings from two models?
No. Each model defines its own space, so a model change means re-embedding everything.
Do they replace keyword search?
No. They handle paraphrase well and exact identifiers badly, so most systems run both.
Terms related to Embeddings
Retrieval by meaning rather than by matching strings, what it is genuinely better at, and the class of query where it reliably fails.
The architecture that retrieves documents at query time and has a model write from them, and the reason your page can be quoted without ever being trained on.
The units a language model actually reads and writes, why they are not words, and why every limit and price you meet is denominated in them.
The token budget a model can reference in one request, what counts against it, and why a bigger window does not remove the need to retrieve selectively.