Reviewed 9 Sept 20265 min

Vector databases: the basics

Vector databases turn meaning into numbers via embeddings, then answer nearest-neighbour queries over them using approximate indexes like HNSW. A five-minute walkthrough of embeddings, similarity search, metadata filtering, and when a vector database is actually the right tool.

Vector databases: the basics

A five-minute read.

Vector databases sound exotic until you see the shape of the problem they solve. Here’s the whole idea.

The problem with keyword search

Traditional databases match exactly. Ask for WHERE title = 'puppy' and you get rows containing the literal string “puppy”. A document about “young dogs” won’t match, even though it’s clearly what you wanted.

Humans search by meaning, not by spelling. That gap is what vector databases exist to close.

Step one: turn meaning into numbers

An embedding model reads a piece of text (or an image, or audio) and outputs a list of numbers, typically a few hundred to a few thousand of them. That list is called a vector, or an embedding.

flowchart LR
    A["puppy"] --> B["embedding model"] --> C["#91;0.12, -0.41, 0.87, ...#93;"]

The numbers themselves are meaningless to look at. What matters is where the vector lands relative to other vectors. Things that mean similar things end up near each other.

flowchart LR
    subgraph S["embedding space"]
        direction LR
        subgraph A["nearby region"]
            a1(puppy)
            a2(kitten)
            a3(wolf)
        end
        subgraph B["nearby region"]
            b1(apple)
            b2(banana)
            b3(mango)
        end
        subgraph C["nearby region"]
            c1(truck)
            c2(van)
            c3(sedan)
        end
    end

Real embedding spaces have hundreds of dimensions rather than two, but the intuition holds: distance means dissimilarity.

To compare two vectors you need a distance measure. The common ones are cosine similarity (the angle between vectors, ignoring their length) and Euclidean distance (straight-line distance). Cosine is the usual default for text.

Step two: store the vectors and search them

A vector database stores those embeddings and answers one main question very fast: given this query vector, which stored vectors are closest?

That’s called nearest neighbour search, and it drives almost everything built on top of vector databases: semantic search, recommendation, deduplication, and retrieval-augmented generation, where relevant documents get pulled in and handed to a language model as context.

The workflow splits into two halves that share one embedding model.

flowchart LR
    D["your content<br/>docs, pages, notes"] --> E["embedding model"]
    E --> F[("vector index<br/>stored, searchable")]

    Q["user question<br/>free text"] --> G["same embedding model"]
    G --> H["similarity search"]
    F --> H
    H --> R["top k results<br/>chunks + metadata"]

The top path runs once per document, or whenever content changes. The bottom path runs on every query. Using the same embedding model on both sides is not optional. Vectors from different models live in different spaces, and comparing them gives noise.

Why a regular database isn’t enough

You could store vectors in Postgres and compute the distance to every row. With ten thousand vectors that works fine. With ten million, checking every one on every query gets slow.

So vector databases use an approximate nearest neighbour index. Instead of scanning everything, they build a structure (usually a navigable graph such as HNSW, or clusters as in IVF) that lets a query jump toward the right neighbourhood in a handful of steps.

flowchart TB
    subgraph BF["brute force: compare the query with every vector"]
        direction TB
        Q1((query)) --> V1((v1))
        Q1 --> V2((v2))
        Q1 --> V3((v3))
        Q1 --> V4((v4))
        Q1 --> V5((v5))
        Q1 --> V6((v6))
    end

flowchart LR
    subgraph ANN["graph index: follow a few hops to a close match"]
        direction LR
        Q2((query)) --> N1((hop 1))
        N1 --> N2((hop 2))
        N2 --> M((match))
        N1 -.- X1((skipped))
        N2 -.- X2((skipped))
    end

The word approximate is the honest part. These indexes trade a small amount of accuracy for a very large speed gain. You might get 98 of the true top 100 neighbours instead of all 100, in a fraction of the time. Most indexes expose a knob that lets you push toward accuracy or toward speed.

Metadata and filtering

Vectors alone are rarely enough. Each one is normally stored with a payload: the original text chunk, a document id, a timestamp, an author, tags.

That lets you combine semantic search with ordinary conditions, like finding the most relevant passages that also belong to a specific customer and were published this year. How well a system handles filtering during search, rather than before or after it, is one of the real differences between products.

When to reach for one

Good fit when relevance is fuzzy and meaning matters more than exact tokens:

question answering over your own documents
“find similar” features
semantic product discovery
duplicate detection
image or audio search

Poor fit for exact lookups, aggregations, transactions, and reporting. Keep using a relational database for those. In practice, most systems run both, and many combine keyword and vector scores into a hybrid ranking, because keyword search still wins for product codes, names, and rare terms.

Three things that bite beginners

Chunk size matters more than people expect. Embedding a whole 50-page document gives you one blurry vector. Splitting it into passages of a few hundred words usually retrieves far better.

Changing your embedding model means re-embedding everything. Old and new vectors are not comparable.

Similarity scores are relative, not absolute. A top result always exists, even when nothing in your data actually answers the question, so plan a threshold or a fallback.