What Are Embeddings in Plain Language?
An embedding is a way to turn text, a phrase, or a document into a set of numbers - a vector of fixed length. An embedding model learns so that semantically similar phrases end up "close" in this numeric space, while different ones stay far apart. That is what powers semantic search, RAG, recommendations, and document clustering. Below - what embeddings are, how they differ from tokens and LLM answers, and where they actually matter for business.
- Vector - a list of hundreds or thousands of numbers, a "fingerprint of meaning" for text
- Embedding model - a separate neural network that encodes text into a vector; not a chat model
- Semantic similarity - "courier delivery" and "express shipping" are closer than "delivery" and "tax return"
- Main use cases - meaning-based search, RAG, deduplication, classification
- Do not confuse - an embedding does not generate an answer; it only helps find relevant text chunks
- In practice - index the knowledge base once, then retrieve Top-K nearest chunks per query
What an Embedding Is in Two Words
Imagine a library where books are shelved not by author name but by topic: logistics texts sit together, accounting separately, marketing in another section. An embedding does the same for text, only in a multidimensional space of numbers.
An embedding is the output of an encoder model: input might be "how to process a product return", output might be a vector of 1,536 numbers. Every chunk of documentation, ticket, or article paragraph gets its own vector. When a user question arrives, it is encoded with the same model and the system looks for the nearest matches.
| Concept | What it is | Example |
|---|---|---|
| Text | Source words | "Delivery time 2-3 days" |
| Embedding model | Encoder neural network | text-embedding-3-small, bge-m3 |
| Vector | Numeric representation | [0.12, -0.04, 0.88, …] |
| Vector DB | Vector storage + fast search | pgvector, Qdrant, Pinecone |
An embedding does not "understand" text like a human and does not write answers. It provides coordinates of meaning so a computer can quickly compare millions of fragments.
Why Vectors If You Already Have Regular Search
Classic keyword search looks for exact matches: a query "return" will not find a paragraph that only says "refund policy" or "reverse shipment process". Semantic search with embeddings compares meaning, not letters.
Typical tasks:
- Corporate FAQ and support - an agent or bot finds the right policy from how the customer phrases the question.
- RAG - before the LLM answers, relevant chunks are pulled from wiki, PDFs, tickets.
- Similar ticket search - "we already had this incident" without manual tagging.
- Deduplication - filter near-duplicate articles, reviews, product cards.
- Clustering - group requests, reviews, forum topics for analytics.
If all needed text fits in a model's context window and changes rarely, embeddings are optional. Once the corpus is large, live, and changing - a vector index is almost always cheaper than feeding everything into the prompt every time.
How Text Becomes Numbers
A simplified pipeline:
Document → chunking → embedding model → vectors → vector DB
User query → same embedding model → query vector → Top-K search → LLM
1. Chunking
Long PDFs or article bases are split into chunks of 256-1,024 tokens with small overlap so paragraphs are not cut mid-thought. Chunks that are too small lose context; chunks that are too large blur search precision.
2. Embedding model
A specialized model runs each chunk and returns a vector of fixed dimension - 384, 768, 1,536, or 3,072 numbers depending on the model. Cloud options: OpenAI text-embedding-3, Cohere, Voyage. Open-source options include BGE, E5, multilingual-e5; you can run them locally via Ollama or your own server.
3. Indexing and search
Vectors go into a vector store - PostgreSQL with pgvector, Qdrant, Weaviate, Chroma. On query, the system measures distance between the question vector and database vectors; most often cosine similarity. It returns the Top-K closest fragments - usually 3 to 10.
Search quality is ~80% determined by "chunking + embedding model", not by picking the smartest chat model. A full breakdown of retrieval + generation is in the RAG article.
Semantic Closeness Without Magic
"Closeness" of vectors is not dictionary synonyms. The model was trained on huge corpora so neighboring points correspond to similar contexts. Therefore:
- "shipping price" and "courier service cost" end up nearby;
- "Python for data analysis" is closer to "pandas tutorial" than to "snake at the zoo";
- on multilingual models, a Russian query can find an English paragraph with the same meaning.
Important: you can compare only vectors produced by the same embedding model and version. Mixing OpenAI and BGE vectors in one index is invalid - they are different "maps".
Where Embeddings Show Up in Products
| Scenario | Role of embeddings |
|---|---|
| Corporate chatbot | Find relevant articles before calling the LLM |
| IDE and code | Semantic repo search (Cursor @Codebase, Copilot indexes) |
| Marketplace | "Similar products", search by description without exact SKU |
| Moderation | Spam clustering, duplicate review detection |
| CRM | Similar deal matching, RAG over knowledge base for managers |
In every case the pattern is the same: retrieval on vectors + reasoning on LLM. Embeddings save context window space and API money: only retrieved chunks go into the prompt, not the whole database.
Embeddings vs Tokens vs Fine-Tuning
Three different mechanisms people often confuse:
| Embeddings | LLM tokens | Fine-tuning | |
|---|---|---|---|
| Task | Meaning search | Text generation | Change model behavior |
| Output | Numeric vector | Text / answer tokens | New model weights |
| Knowledge updates | Re-index documents | New prompt / RAG | Retraining |
| Cost | Cheaper than generation | Pay for input + output | Expensive and slow |
A token is a billing and slicing unit for chat models. An embedding is a separate API product with different per-1M pricing and a different model. Compare strategies "everything in context", RAG, and fine-tuning in context window vs RAG vs fine-tuning.
How to Choose an Embedding Model
In practice, look at four parameters:
- Languages - Russian and mixed corpora need multilingual models (bge-m3, multilingual-e5, voyage-3).
- Dimension - more dimensions are often more accurate but heavier indexes and storage.
- Latency and cost - for millions of documents, estimate primary indexing and re-indexing on updates.
- Deployment - cloud API is easier to start; local models for private data without sending to SaaS.
For an MVP, one proven model and pgvector in existing PostgreSQL is often enough. Adding multiple rerankers and hybrid BM25 + vectors makes sense when search metrics already hit a ceiling.
Typical Mistakes
- Indexing carelessly - no HTML cleanup, no chunking strategy, duplicates - noisy search.
- Changing model without re-indexing - old and new vectors are incompatible.
- Expecting reasoning from embeddings - a vector will not answer; you need an LLM or templates on top of retrieved chunks.
- Ignoring metrics - without a test set "question → expected document", gut-feel tuning rarely survives production.
- Confusing with full-text search - for SKUs, part numbers, exact codes, a hybrid of keywords + embeddings works better.
Summary
An embedding is a numeric "fingerprint of meaning" for text. You need it where finding the relevant piece among a large corpus matters: RAG, support, internal wikis, code search. For a short chat with one file in context, a vector index is optional. For a live knowledge base, embeddings are a basic brick; the LLM is built on top, not instead of them.
Frequently Asked Questions
How is an embedding different from a token?
A token is a text fragment for chat models: it drives prompt length and API cost. An embedding is one vector for an entire text fragment (sentence, paragraph, chunk) that captures meaning for search. Tokens slice strings for generation; embeddings encode meaning for comparison with other fragments.
Why a separate embedding model if ChatGPT and Claude exist?
Chat models are optimized for generating answers. An embedding model is optimized for compressing meaning into a compact vector and fast search. Dedicated encoders are cheaper at large indexing volumes, more stable for retrieval, and do not burn expensive output tokens on "summarizing" documents.
Can you mix vectors from different models in one database?
No. Each model has its own coordinate space and dimension. Build the index with one embedding model; when you change models - full re-indexing. Compare only vectors from the same encoder in the same version.
How much do embeddings cost via API?
Usually cheaper than text generation from the same provider: billing is for input tokens on the embedding endpoint, without output. Exact price depends on model and volume; for a monthly budget estimate, multiply corpus token count by the per-1M rate and plan for re-indexing on updates. More on token logic and calculation in token pricing.
Do a simple website chatbot need embeddings?
Not always. If the bot answers from a fixed FAQ of a dozen pages or loads one PDF into the context window once - a prompt is enough. Embeddings are needed when there are many documents, they change often, and search by user phrasing matters - the typical RAG case for support and internal assistants.