Vector search
Lesson 4 of 10
4Vector search
The big pictureVector search turns each document into a vector once, then turns the question into a vector and finds the nearest documents. It finds "when do I get my travel allowance?" in a paragraph about "paying allowances" with almost no word in common. In this lesson you also see how search is sped up over millions of vectors, and where similarity misleads you.
What the picture shows: The narrow angle between the question and the allowance document means a similarity of 0.90, and the leave carry-over document sits wider at 0.64. Notice the second does not answer at all: similarity brings together what looks or is used alike, not what answers the question.
Map of this part
The concepts in order, click any one to jump to it:
Two ways into the same idea
Ordinary search finds only the same word, so asking about "car" will not find "vehicle". Vector search turns every text into an address on the map of meaning, then finds the texts nearest your question on that map, even if they share not a single word.
You embed passages with an embedding model and store them in pgvector in a vector(n) column, then search nearest neighbours by cosine distance <=> with an approximate HNSW index. Similarity brings the topic close, not the truth, so it can return a passage on the same topic with wrong or negated numbers.
Before You Start
🧭Embeddings
You know cosine, and that normalised vectors only need a dot product.
🧩Clustering and dimensionality
You know k-means, which is the idea behind the IVF index.
The Core Idea
Vector search finds meaning even with no words in common. In this lesson you build a complete search, see where similarity misleads you, speed it up with an IVF index, then write it in pgvector.
1From text to vector to result
Each document becomes a vector once, at indexing. At search time the question becomes a vector, and the results are the documents with the highest cosine similarity.
Look at the picture: the heavy work happens once, and each search is one dot product.
import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.decomposition import TruncatedSVD
docs = [
"يحق للموظف إجازة سنوية مدتها واحد وعشرون يوم عمل في السنة.",
"تقدم طلبات الإجازة السنوية قبل أسبوعين على الأقل من موعدها.",
"الإجازة المرضية تحتاج تقريرا طبيا إذا زادت عن يومين متتاليين.",
"يوافق المدير المباشر على الإجازة المرضية والسنوية.",
"يمكن ترحيل عشرة أيام من الإجازة السنوية إلى السنة التالية فقط.",
"تصرف بدلات السفر خلال عشرة أيام من تقديم الفواتير.",
"يجب حجز تذاكر السفر عبر بوابة الحجز الداخلية.",
"ساعات العمل من الثامنة صباحا حتى الرابعة عصرا.",
"العمل عن بعد مسموح يومين في الأسبوع بموافقة المدير.",
"رمز الخطأ E-1043 في بوابة الحجز يعني انتهاء الجلسة.",
]
# a stand-in for an embedding model: character n-grams compressed to 8 dimensions
tfidf = TfidfVectorizer(analyzer="char_wb", ngram_range=(2, 4))
svd = TruncatedSVD(n_components=8, random_state=0)
vectors = svd.fit_transform(tfidf.fit_transform(docs))
vectors /= np.linalg.norm(vectors, axis=1, keepdims=True)
def search(query, k=3):
q = svd.transform(tfidf.transform([query]))[0]
q /= np.linalg.norm(q)
scores = vectors @ q
top = np.argsort(-scores)[:k]
return [(int(i), round(float(scores[i]), 2)) for i in top]
for query in ["كم يوم إجازة سنوية لي؟", "متى أستلم بدل السفر؟"]:
print(query)
for i, s in search(query):
print(f" {s:.2f} [{i}] {docs[i]}")
The first question found the annual leave document at 0.89, the second the allowance document at 0.90. Look at second place for the second question: the leave carry-over document (0.64), because it shares "ten days" with the allowance document. Similarity brings together what looks or is used alike, not necessarily what answers the question.
2Where similarity misleads
Similarity brings together what looks or is used alike, not necessarily what answers. Look at the picture: for the annual leave question, the carry-over document came second at 0.74, and it does not answer how many days.
Similarity also confuses opposites, and weakens on codes, numbers and rare names like E-1043. That is why we add keyword search in the hybrid lesson.
3An approximate index: IVF
Exhaustive search compares the question with every vector. It is fully exact and enough up to about a hundred thousand documents. Beyond that you need an index that checks a small part and finds most of the nearest.
Look at the picture: the vectors gather around centres. At search time you check only the clusters nearest the question, and their number is called nprobe.
Twenty thousand 64-dimensional vectors split into 100 clusters. We measure recall at 10 against exhaustive search, by number of clusters probed.
import numpy as np
from sklearn.cluster import KMeans
rng = np.random.default_rng(0)
centers = rng.normal(size=(50, 64))
base = centers[rng.integers(0, 50, 20000)] + rng.normal(scale=1.5, size=(20000, 64))
base /= np.linalg.norm(base, axis=1, keepdims=True)
queries = base[rng.choice(20000, 100, replace=False)] + rng.normal(scale=0.05, size=(100, 64))
def exact(q, k=10):
return set(np.argsort(-(base @ q))[:k])
km = KMeans(n_clusters=100, n_init=1, random_state=0).fit(base)
lists = [np.where(km.labels_ == c)[0] for c in range(100)]
def ivf(q, k=10, nprobe=5):
near = np.argsort(-(km.cluster_centers_ @ q))[:nprobe]
cand = np.concatenate([lists[c] for c in near])
return set(cand[np.argsort(-(base[cand] @ q))[:k]]), len(cand)
truth = [exact(q) for q in queries]
for nprobe in [1, 5, 20]:
recall, scanned = 0, 0
for q, t in zip(queries, truth):
found, n = ivf(q, nprobe=nprobe)
recall += len(found & t) / 10
scanned += n
print(f"nprobe={nprobe:2}: recall@10 = {recall / 100:.2f}, vectors scanned = {scanned / 100 / 20000:.0%} of all")
Probing one cluster (1% of the vectors) finds 63% of the true nearest. Five clusters (5%) find 94%: about twenty times faster for a 6% loss. This is the knob you will tune in any vector database: in pgvector it is ivfflat.probes, in HNSW ef_search. Tune it by measuring recall on your questions, not by the default value.
HNSW is another way: it builds a graph of neighbours the search hops across. Very fast and accurate, using more memory.
4Vectors inside Postgres
pgvector is a PostgreSQL extension that adds a vector column, distance operators and indexes. Your documents, their permissions and their vectors stay in one database, an excellent start for most projects.
This is what you will actually write in Postgres. It needs a database with the pgvector extension, so it is here for reading.
CREATE EXTENSION IF NOT EXISTS vector; CREATE TABLE chunks ( id bigserial PRIMARY KEY, doc_id text NOT NULL, body text NOT NULL, model text NOT NULL, -- which embedding model made this vector embedding vector(384) NOT NULL ); CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops); -- $1 is the question's vector; <=> is cosine distance, so smaller is closer SELECT doc_id, body, 1 - (embedding <=> $1) AS similarity FROM chunks WHERE model = 'multilingual-e5-small' ORDER BY embedding <=> $1 LIMIT 5;
<=> is a distance, not a similarity, so we sort ascending and convert with 1 − for display. The model column stops vectors from two models mixing. And pgvector's key advantage: you can add a WHERE on permissions, department or date in the same query, which the system limits lesson relies on.
How It Works, Step by Step: how an IVF index finds the nearest documents
At indexing you split all vectors into clusters with k-means, 100 clusters in the twenty-thousand example.
At search time you compare the question's vector with the cluster centres only, and pick the nearest nprobe of them.
You compare the question with the vectors inside those clusters alone, and return the highest similarities.
You measure recall against exhaustive search on your questions, and tune nprobe until the balance of accuracy and speed suits you.
Worked example
Ten short documents from company policies. We turn them into vectors, normalise them and search with dot products. Since real embedding models do not run in the browser, we use a small stand-in: character n-grams compressed to eight dimensions. The idea, and the search code around it, are the same with any model.
Try it without code
Myth vs Reality
| Myth | Reality |
|---|---|
| Every search project needs a specialised vector database. | Up to about a hundred thousand documents exhaustive search with one matrix product is enough, and pgvector keeps vectors beside your data and permissions. |
| An approximate index gives the same results as exhaustive search. | It gives up some accuracy: one cluster found 63% of the true nearest, five clusters 94%. |
| High similarity means the document answers the question. | The carry-over document scored 0.64 for the allowance question only because it shares "ten days". Similarity measures closeness, not answering. |
Try it yourself
You raise nprobe from 5 to 20 in an IVF index. What happens?
You check more clusters, so you compare more vectors: recall rises and speed falls. That is what you saw: from 94% to 98% for four times the work.
Why store the embedding model's name with every vector?
Each model is its own space. When you move to a better model, this column tells you what has not been re-embedded yet, and stops search from mixing the two.
Quick review cards
Check Your Understanding
You have only 5,000 documents. Do you need an HNSW index?
Usually not. Comparing 5,000 vectors of 384 dimensions takes thousandths of a second and is 100% exact. The index adds complexity and a small accuracy loss for little benefit here. Add it when you measure real slowness.
In the pgvector query, why sort ascending by <=> and then show 1 - (embedding <=> $1)?
Because <=> is cosine distance and smaller is closer, so ascending order puts the nearest first. Subtracting it from 1 turns it back into a familiar similarity for display.
Resources for this lesson
Grouped by type. Start with the official docs for precision, or an article or video for the explanation.
Official docs2
Done with this lesson?
Sign in to save your progress.