Vector Databases

A vector database stores embeddings and quickly finds the ones closest to a question, even among millions of items.

Why a Special Database?

In RAG Step by Step we kept our embeddings in a NumPy array and compared the question with every chunk. That is fine for a few thousand chunks. But a real shop catalogue, a law library or years of support chats can have millions.

A vector database helps with:

Need	What the database does
Speed	Uses special indexes to find close vectors without checking every row
Saving	Keeps vectors on disk, so you do not re-embed after every restart
Metadata	Stores extra fields like category, date, price or language with each vector
Filtering	"Find similar items, but only in the Shoes category and under Rs 5,000"
Updates	Add, change and delete items while the app is running

How It Works, Built by Hand

Every vector database does the same basic job: store vectors with an id and text, then return the closest ones to a query vector. Here is that job in a few lines of NumPy, using toy 3-number vectors ([phones, clothes, delivery]):

Example

python

import numpy as np

class TinyVectorStore:
    def __init__(self):
        self.ids, self.texts, self.vectors = [], [], []

    def add(self, doc_id, text, vector):
        v = np.array(vector, dtype=float)
        self.ids.append(doc_id)
        self.texts.append(text)
        self.vectors.append(v / np.linalg.norm(v))   # store normalised

    def search(self, query_vector, k=2):
        q = np.array(query_vector, dtype=float)
        q = q / np.linalg.norm(q)
        scores = np.stack(self.vectors) @ q          # cosine for every row at once
        best = np.argsort(scores)[::-1][:k]
        return [(self.ids[i], round(float(scores[i]), 3), self.texts[i]) for i in best]

store = TinyVectorStore()
# Toy vectors: [phones, clothes, delivery]
store.add("p1", "Phone covers in all colours", [0.9, 0.1, 0.0])
store.add("p2", "Lawn suits for summer",       [0.0, 0.95, 0.1])
store.add("p3", "Free delivery in Karachi",    [0.1, 0.1, 0.9])
store.add("p4", "Fast chargers and cables",    [0.85, 0.0, 0.2])

for row in store.search([0.8, 0.0, 0.3]):     # "something for my mobile, delivered"
    print(row)

Output

('p4', 0.992, 'Fast chargers and cables')
('p1', 0.931, 'Phone covers in all colours')

Because every vector is normalised (made length 1), a simple dot product gives the cosine similarity. Real databases do the same maths, plus clever indexes so they do not have to check every row.

Approximate Search

With millions of vectors, comparing with every one is too slow. Vector databases use approximate nearest neighbour (ANN) indexes. One popular kind is called HNSW: it links each vector to some of its neighbours, like a map of shortcuts, and walks across the map towards the closest ones.

"Approximate" means it might very rarely miss the true best match, in return for being much faster. For most apps that trade is worth it.

Popular Choices

Option	Good for
PostgreSQL with pgvector	You already use PostgreSQL and want vectors next to your normal data
Chroma	Learning and small Python projects, runs on your laptop
Qdrant, Weaviate, Milvus	Dedicated vector search, can run on your own server
Managed cloud services	You do not want to run servers yourself
SQLite extensions	Small offline or mobile apps

You do not need the "best" one. For a first project, use what fits your current stack.

Chroma in Python

Chroma (pip install chromadb) can create embeddings for you with a built-in default model. The first run downloads that model, so it needs the internet once:

Example

python

import chromadb

client = chromadb.PersistentClient(path="./shop_db")   # saved in a folder
products = client.get_or_create_collection("products")

products.add(
    ids=["p1", "p2", "p3"],
    documents=["Phone covers in all colours", "Lawn suits for summer", "Fast chargers and cables"],
    metadatas=[{"category": "mobile"}, {"category": "clothes"}, {"category": "mobile"}],
)

result = products.query(
    query_texts=["something to charge my phone quickly"],
    n_results=2,
    where={"category": "mobile"},      # filter by metadata
)
print(result["documents"][0])

It should print the chargers first. The where filter keeps only items in the mobile category.

PostgreSQL with pgvector

If your app already has a PostgreSQL database, pgvector adds a vector column type. Here the vectors have 3 numbers to keep the example short; real ones have as many numbers as your embedding model returns:

Example

sql

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE products (
  id SERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  price_rs INTEGER,
  embedding vector(3)
);

INSERT INTO products (name, price_rs, embedding) VALUES
  ('Phone covers in all colours', 500, '[0.9, 0.1, 0.0]'),
  ('Lawn suits for summer', 4500, '[0.0, 0.95, 0.1]'),
  ('Fast chargers and cables', 1800, '[0.85, 0.0, 0.2]');

-- <=> is cosine distance: smaller means more similar
SELECT name, price_rs
FROM products
WHERE price_rs < 5000
ORDER BY embedding <=> '[0.8, 0.0, 0.3]'
LIMIT 2;

The result is the chargers, then the phone covers. Normal SQL filters (WHERE price_rs < 5000) work together with similarity search. See the PostgreSQL Tutorial for SQL basics.

Store the original text and useful metadata (source file, page, date) next to every vector. The vector alone cannot be turned back into text, and you will need the source to show citations.

Exercise

What is the main job of a vector database in a RAG app?

To generate the final answerTo train the language modelTo store embeddings and quickly find the ones most similar to the questionTo translate documents into Urdu