What you'll learn
Quick Answer
A vector database is purpose-built storage and indexing infrastructure for embeddings — it answers 'find the 10 vectors closest to this one' out of millions of entries in milliseconds, using approximate nearest neighbor algorithms like HNSW instead of comparing against every row. That's the piece that makes RAG, semantic search, and recommendation systems fast enough to run in production. Pinecone, Weaviate, Chroma, and Postgres with the pgvector extension are the four you'll run into most, and they trade off differently on hosting, filtering, and how much you have to operate yourself.
What a Vector Database Actually Stores
An embedding model turns a piece of text, an image, or a chunk of audio into a list of numbers — a vector, usually somewhere between 384 and 3072 numbers long, where similar meanings end up as nearby points in that space. That part is covered in our Vector Embeddings Explained post. This one is about what happens after you have millions of those vectors and need to find the closest ones to a new query, fast, without scanning every single one.
A vector database is not just "a database that happens to hold arrays of floats." You could technically store embeddings as a JSON column in any database. What makes something a vector database is that it ships with an index structure built specifically for similarity search — finding the nearest neighbors of a query vector by distance, not by exact match — plus the surrounding machinery to keep that index correct as data is added, updated, and deleted, and to combine similarity search with normal filters like tenant_id = 42 or published = true.
It sits next to your regular database, not instead of it. Most production RAG systems still keep the source-of-truth text, user data, and permissions in Postgres or MongoDB, and use the vector store purely for the "which chunks are semantically closest to this query" lookup. Some products, like pgvector, collapse both roles into one system; others, like Pinecone, are dedicated services you call over an API.
Brute-Force Search and Why It Breaks at Scale
Before talking about indexes, it's worth seeing what they're replacing. If your embeddings are normalized to unit length, cosine similarity between two vectors is just their dot product. Finding the nearest neighbors of a query is: compute that dot product against every stored vector, then take the top few. Here's that exact brute-force search, run against 5,000 random 384-dimensional unit vectors (the same dimensionality as a small sentence-transformer model):
rng = np.random.RandomState(3)
vectors = rng.normal(size=(5000, 384)).astype(np.float32)
vectors /= np.linalg.norm(vectors, axis=1, keepdims=True)
query = rng.normal(size=(384,)).astype(np.float32)
query /= np.linalg.norm(query)
sims = vectors @ query
top5 = np.argsort(-sims)[:5]Running this gave top-5 neighbor IDs [2152, 702, 420, 4789, 2049] with cosine scores [0.1924, 0.1844, 0.1608, 0.1595, 0.1582] (low scores are expected here since the vectors are random noise, not real embeddings — the point is the mechanism, not the scores). The search itself took about 0.65-0.8 ms averaged over repeated runs.
That sounds fine, until you check how it scales. The same brute-force search against 50,000 vectors took 4.5 ms on the same machine — roughly proportional to the number of vectors, as you'd expect from an operation that touches every row. Extend that line and a 50-million-vector collection, the kind of scale a real production RAG system with years of documents can reach, would take multiple seconds per query using brute force alone. That's fine for a nightly batch job and unusable for a live search bar or a chat app waiting on an answer. This is the actual, specific problem vector database indexes exist to solve: trading a small amount of accuracy for a large amount of speed.
How HNSW Indexing Actually Works
HNSW (Hierarchical Navigable Small World) is the index behind most modern vector databases — Pinecone, Weaviate, Qdrant, and pgvector all support it. The idea is to build a graph where each vector is a node connected to a handful of its approximate neighbors, arranged in layers. The top layer is sparse with long-range links; each layer below is denser with shorter-range links, down to the bottom layer which contains every vector.
A search starts at an entry point in the top layer and greedily walks toward the query — at each step, jump to whichever connected neighbor is closest to the query, until no neighbor is closer than the current node. Then drop down a layer and repeat, using the previous layer's result as the new starting point. By the time you reach the bottom layer, you've narrowed down to a small neighborhood of genuinely close candidates without ever touching most of the graph. It's the same intuition as a highway system: long-haul roads at the top layer get you roughly to the right city fast, then progressively local roads get you to the right street.
This is approximate, not exact — the graph doesn't guarantee it finds the true nearest neighbor, only a very likely one, which is why the metric people report for these indexes is recall (what fraction of true top-k results the approximate search actually returns) traded against query latency. Two parameters control that tradeoff directly: m, how many neighbor connections each node keeps (higher m means a more thorough graph, more memory, slower to build), and ef_construction/ef_search, how wide a candidate list the algorithm explores while building or querying (higher means better recall, slower). In pgvector, building one looks like this:
CREATE INDEX ON items USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);
SET hnsw.ef_search = 100;
SELECT id FROM items ORDER BY embedding <=> '[0.1, 0.2, ...]' LIMIT 10;Note the operator: <=> is cosine distance, <-> is L2/Euclidean distance, and <#> is negative inner product — pgvector only uses the index if your query's distance operator matches the operator class the index was built with.
IVF and Quantization: the Other Index Family
HNSW isn't the only approach. IVF (Inverted File index) takes a completely different strategy: run k-means clustering over your vectors once, ahead of time, to group them into a fixed number of clusters ("lists"), each with a centroid. At query time, instead of searching everything, you first find the handful of cluster centroids closest to your query, then only search inside those clusters. If you have 1,000 lists and probe the nearest 10, you've cut the search space by roughly 99% before doing a single fine-grained comparison.
The tradeoff against HNSW is real and specific: IVF indexes are cheaper to build and use less memory, but recall depends heavily on how many lists you probe (nprobe), and vectors that sit near a cluster boundary can get missed if their true nearest neighbors live in an adjacent, unprobed cluster. HNSW generally gives better recall at a given latency, but costs more memory because every vector carries a set of graph edges, and index builds are slower. pgvector supports IVFFlat as its older, lighter-weight option and HNSW as the newer, generally-preferred one for most workloads since pgvector 0.5.0.
Product quantization (PQ) is a separate, complementary trick that several products layer on top of either index: instead of storing the full-precision vector, split it into sub-vectors and replace each sub-vector with the ID of its nearest centroid from a small codebook, shrinking a 1536-dimension float32 vector (6 KB) down to a few hundred bytes. You lose some precision, which is why PQ is usually paired with a reranking step that re-scores the top candidates using the original full-precision vectors before returning results.
Pinecone, Weaviate, Chroma, and pgvector: Picking One
Pinecone is a fully managed, serverless vector database — you don't choose an index type or tune HNSW parameters yourself; you create an index, pick a similarity metric, and Pinecone handles sharding, scaling, and index maintenance behind its API. A basic upsert and query with the current Python SDK looks like this:
from pinecone import Pinecone
pc = Pinecone(api_key="your-api-key")
index = pc.index("docs")
index.upsert(vectors=[
("chunk-1", [0.012, -0.087, 0.153, ...]),
("chunk-2", [0.045, 0.021, -0.064, ...]),
], namespace="my-app")
results = index.query(vector=[0.012, -0.087, 0.153, ...], top_k=5, namespace="my-app")
for match in results.matches:
print(match.id, match.score)It's the fastest path to production if you don't want to run infrastructure, and namespaces give you a clean way to isolate tenants. The tradeoff is that it's a separate system from your primary database, billed by stored vectors and query volume, so it adds both an operational dependency and a recurring cost line.
Weaviate is closer to a full document database that happens to be vector-native — objects carry a real schema (typed properties, not just an ID and a blob), it has built-in hybrid search that blends BM25 keyword scoring with vector similarity in one query, and it can be self-hosted or used as a managed cloud service. It's the strongest default when you need genuine hybrid search or want to keep the option of self-hosting.
Chroma is the developer-first option — it started as an embedded, local-first database you could run with zero setup for prototyping (pip install chromadb and go), and has since added a managed Chroma Cloud offering for taking the same collections to production. It's the easiest one to get a RAG prototype running with in an afternoon:
import chromadb
client = chromadb.PersistentClient(path="./chroma_data")
collection = client.get_or_create_collection("docs")
collection.add(
documents=["Photosynthesis converts light into chemical energy."],
metadatas=[{"source": "textbook"}],
ids=["doc-1"],
)
results = collection.query(query_texts=["how do plants make energy"], n_results=5)pgvector is a Postgres extension, not a separate database at all. If you already run Postgres — and most backends do — adding vector search means CREATE EXTENSION vector;, a new column of type vector(1536), and an HNSW index, with zero new infrastructure to operate, and the ability to join vector search against your actual relational data in a single SQL query. The tradeoff shows up at very large scale or very high query throughput, where a system purpose-built for vector workloads and horizontal scaling can outperform a general-purpose database with an extension bolted on. For most applications under tens of millions of vectors, that gap doesn't matter enough to justify running a second system.
Metadata Filtering and Hybrid Search
Real queries are almost never "find the closest vectors" in isolation — they're "find the closest vectors where tenant_id = 42 and status = 'published'." How a vector database handles that combination matters more than raw index speed in practice, and there are two fundamentally different strategies.
Pre-filtering applies the metadata filter first, then runs similarity search only within the surviving rows. This gives exact filter correctness but can defeat the whole point of an approximate index if the filter is very selective — searching within 200 matching rows out of 10 million doesn't benefit much from a graph built for the full 10 million. Post-filtering runs the ANN search first, gets its top-k approximate neighbors, and then discards any that fail the metadata filter — which is fast, but if the filter is selective enough, you can end up with fewer than k results, or none, because none of the top candidates happened to match. Weaviate, Pinecone, and pgvector each have their own middle-ground heuristics here (Pinecone applies filters during the graph traversal itself rather than strictly before or after), and it's worth explicitly testing filtered queries at your real selectivity, not just unfiltered ones, before assuming an index configuration works.
Hybrid search — combining a keyword score (typically BM25) with a vector similarity score, then merging the two rankings, usually with reciprocal rank fusion — exists because pure vector search has a real weakness: it's bad at exact terms. A query for an error code, a product SKU, or a specific person's name can retrieve semantically related but factually wrong chunks, because the embedding model captures topic and tone more reliably than exact tokens. Weaviate and modern Pinecone both support hybrid search natively; with pgvector you'd combine it yourself using Postgres's built-in full-text search (tsvector) alongside the vector column.
Running One in Production
A few operational realities only show up after the prototype works. First: if you ever change your embedding model — upgrading to a newer, better one, or switching providers — every vector in your store becomes incompatible with new queries, because different models place meaning in different, non-comparable coordinate spaces. There is no in-place migration; you have to re-embed your entire corpus and rebuild the index from scratch, which for a large document set is a real batch job with real API or compute cost, not a config change.
Second, recall isn't free and isn't fixed — ef_search (HNSW) or nprobe (IVF) are knobs you tune per query pattern, and the right value depends on how much wrong-answer tolerance your application has. A recommendation feed can tolerate more approximation than a RAG system answering factual questions, where a missed relevant chunk means a wrong or incomplete answer with no visible error.
Third, cost scales with both vector count and dimensionality, and dimensionality is easy to overspend on. A 3072-dimension embedding isn't automatically better for your use case than a 768-dimension one — it's more expensive to store, slower to index, and the accuracy gain is often marginal for a specific domain. It's worth actually testing a smaller embedding model against your real data before defaulting to the largest one available. Finally, treat your index the way you'd treat any other production data store: back up the underlying vectors and metadata somewhere index-independent, so a corrupted or misconfigured index is a rebuild, not a data loss incident.