Skip to content

What Are Embeddings in AI? Vectors, Semantic Search and Vector Databases Explained

Embeddings turn words, documents and images into lists of numbers that capture meaning, so computers can find similar things. How embeddings work, cosine similarity, vector databases, how RAG uses them, and how much memory they need.

5 min read

What Are Embeddings? — cover image from TensorCore.si

Short answer

An embedding is a list of numbers (a vector) that represents the meaning of a piece of data such as a word, sentence, document, image or product. Embedding models are trained so that similar things get similar vectors, which lets computers find related content by comparing numbers instead of matching exact keywords. Embeddings power semantic search, recommendations, clustering and retrieval-augmented generation (RAG), and they are usually stored and searched in a vector database.

The idea: meaning as coordinates

Imagine placing every word on a map so that words with similar meanings sit close together: "cat" near "kitten", "GPU" near "graphics card", "Paris" near "France". An embedding is a position on such a map, except the map has hundreds or thousands of dimensions instead of two.

For example, a sentence might become a vector like this (shortened):

"How much VRAM do I need for Llama?"  →  [0.021, -0.113, 0.087, ..., 0.045]   (1,536 numbers)

The individual numbers have no simple human meaning. What matters is how close vectors are to each other.

Measuring similarity

The most common measure is cosine similarity, which compares the angle between two vectors:

  • Close to 1: very similar meaning
  • Around 0: unrelated
  • Negative: opposite in some sense (rare in practice for text embeddings)

Other measures are dot product and Euclidean (L2) distance. Many embedding models output normalised vectors, in which case cosine similarity and dot product give the same ranking.

Where embeddings come from

Embeddings are produced by neural networks trained so that related items end up close together:

  • Word2vec (2013) learned word vectors by predicting nearby words. It became famous for vector arithmetic such as king − man + woman ≈ queen.
  • Transformer encoders such as BERT and Sentence-BERT produce embeddings for whole sentences and paragraphs, taking context into account, so "bank" in "river bank" and "bank account" get different representations.
  • Modern embedding models from OpenAI, Google, Cohere, Voyage and open-source projects (such as the BGE, E5, GTE, Nomic and Qwen embedding families) are trained on huge sets of matching pairs, like questions and their answers.
  • Multimodal models such as CLIP place images and text in the same space, so the text "a red sports car" lands near photos of red sports cars.

Inside every large language model, the first step is also an embedding: each token ID is turned into a vector before the transformer layers process it.

What embeddings are used for

  • Semantic search: find documents that match the meaning of a query, even without the same words. A search for "GPU running out of memory" can find a page titled "Fixing CUDA OOM errors".
  • RAG: retrieve the most relevant passages and give them to a chatbot so it answers from real sources, reducing hallucinations.
  • Recommendations: suggest products, articles or songs similar to ones a user liked.
  • Clustering and topic discovery: group support tickets, reviews or articles by theme.
  • Duplicate detection: find near-identical questions or pages.
  • Classification: use embeddings as input features for a small, fast classifier.
  • Anomaly detection: flag items whose vectors are far from everything else.

How embeddings power RAG, step by step

1. Chunk your documents into passages of a few hundred tokens. 2. Embed every chunk with an embedding model and store the vectors with the original text. 3. When a user asks a question, embed the question with the same model. 4. Search for the chunks whose vectors are most similar to the question vector. 5. Insert the top results into the language model's context window and ask it to answer using them, with citations.

Many systems combine vector search with traditional keyword search (hybrid search) and re-rank the results with a second model for better accuracy. For the full picture see What is RAG?.

Vector databases

Comparing a query against millions of vectors one by one is slow, so vector databases use approximate nearest neighbour (ANN) indexes, such as HNSW graphs or IVF clustering, that find very close matches in milliseconds while checking only a small fraction of vectors.

Popular options:

ToolTypeGood for
pgvectorExtension for PostgreSQLAdding vector search to an existing Postgres database
FAISSOpen-source library (Meta)Fast in-process search, GPU acceleration
Qdrant, Weaviate, Milvus, ChromaOpen-source vector databasesDedicated self-hosted or managed vector search
PineconeManaged cloud serviceServerless vector search without running infrastructure
Elasticsearch / OpenSearchSearch engines with vector supportHybrid keyword plus vector search

How much memory do embeddings need?

Storage is simple to estimate: number of vectors × dimensions × bytes per number.

VectorsDimensionsFormatRaw size
1 million768FP32 (4 bytes)about 3.1 GB
1 million1,536FP32about 6.1 GB
1 million3,072FP32about 12.3 GB
1 million1,536INT8 (1 byte)about 1.5 GB
1 million1,536Binary (1 bit)about 0.19 GB

Indexes add overhead on top. Common ways to cut costs:

  • Fewer dimensions: some models, including OpenAI's text-embedding-3 models, let you shorten vectors with only a small loss of accuracy.
  • Quantization: store vectors as INT8 or binary, the same idea as model quantization.
  • GPU acceleration: libraries such as FAISS and NVIDIA cuVS can build and search indexes on GPUs, which helps with very large collections.

Creating embeddings for millions of documents is also a good fit for GPUs, because embedding models are small and process text in large parallel batches.

How to choose an embedding model

  • Check benchmarks like the MTEB leaderboard on Hugging Face, but test on your own data.
  • Match your languages: use a multilingual model if your content is not only in English.
  • Consider dimensions and cost: larger vectors can be more accurate but cost more to store and search.
  • Check the maximum input length so your chunks fit.
  • Never mix models: queries and documents must be embedded with the same model. Changing models means re-embedding everything.

Frequently asked questions

Are embeddings the same as tokens?

No. Tokens are the pieces text is split into, each with an ID number. Embeddings are dense vectors of many numbers that represent meaning. Language models convert tokens into embeddings as their first step.

What is the difference between a vector and an embedding?

A vector is any list of numbers. An embedding is a vector produced by a model specifically to represent the meaning or features of some data.

Do I need a vector database?

Not always. For a few thousand documents, an in-memory array or pgvector in an existing database is enough. Dedicated vector databases help when you have millions of vectors, need filtering at scale or want managed infrastructure.

Can embeddings work for images and audio?

Yes. Image, audio and multimodal embedding models place these data types in vector spaces too, enabling reverse image search, finding similar sounds and searching photos with text.

Sources and further reading

Think you understand it now? Prove it.

Solve real Tensor Core math in the TensorCore.si Challenge and earn a certificate with your name.

Take the challenge

Leave a Reply

Your email address will not be published. Required fields are marked *