Embeddings and Vectors Explained Simply

An embedding is a list of numbers that describes the meaning of a piece of text, so a computer can measure how similar two texts are.

From Words to Numbers

Computers are good with numbers, not meanings. So we turn text into a vector, which is simply a list of numbers like [0.12, -0.80, 0.33, ...].

An embedding model does this job. You send it a sentence and it returns a vector, often with hundreds or thousands of numbers. The model was trained so that:

• Texts with similar meaning get vectors that point in a similar direction
• Texts with different meaning get vectors that point in different directions

"How much is the fee?" and "What does the course cost?" share almost no words, but their embeddings are close. This is why embeddings beat simple keyword search.

A Picture with Only Three Numbers

Real embeddings have many numbers and we cannot say what each one means. To build intuition, imagine tiny vectors where we choose the meaning of each number ourselves: [food, transport, money].

Biryani and nihari point in nearly the same direction. Rickshaw points somewhere else.

Measuring Similarity: Cosine Similarity

Cosine similarity measures how closely two vectors point in the same direction:

• 1 means the same direction (very similar meaning)
• 0 means unrelated
• Negative values mean opposite directions

The formula is the dot product of the two vectors divided by the product of their lengths. In NumPy it is one line:

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Toy 3-number "embeddings": [food, transport, money]
biryani = np.array([0.9, 0.1, 0.2])
nihari  = np.array([0.8, 0.0, 0.3])
rickshaw = np.array([0.1, 0.9, 0.2])

print("biryani vs nihari:  ", round(cosine_similarity(biryani, nihari), 3))
print("biryani vs rickshaw:", round(cosine_similarity(biryani, rickshaw), 3))

biryani vs nihari:   0.984
biryani vs rickshaw: 0.256

If NumPy is new to you, see the NumPy Tutorial.

Searching by Meaning

Now the useful part. Store an embedding for each of your documents. When a question comes, embed the question and find the closest documents:

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))

# Toy embeddings: [fees, timing, location, computers]
documents = {
    "The monthly fee is paid at the office.":        np.array([0.9, 0.1, 0.2, 0.0]),
    "Evening classes start at 6 pm.":                np.array([0.1, 0.9, 0.1, 0.1]),
    "The institute is near the main road in Orangi.": np.array([0.0, 0.1, 0.9, 0.0]),
    "Every student gets their own computer.":        np.array([0.1, 0.0, 0.1, 0.9]),
}
# Imagine the model turned "How much do I have to pay?" into this vector:
question = np.array([0.8, 0.2, 0.1, 0.1])

scores = {doc: cosine_similarity(question, vec) for doc, vec in documents.items()}
for doc, score in sorted(scores.items(), key=lambda x: x[1], reverse=True):
    print(f"{score:.3f}  {doc}")

0.980  The monthly fee is paid at the office.
0.365  Evening classes start at 6 pm.
0.236  Every student gets their own computer.
0.145  The institute is near the main road in Orangi.

The question "How much do I have to pay?" does not contain the word "fee", but the fee document wins. That is semantic search.

Getting Real Embeddings

In a real app you call an embeddings endpoint. With an OpenAI-compatible provider it looks like this (pip install openai):

import os
from openai import OpenAI

client = OpenAI(base_url=os.environ["AI_BASE_URL"], api_key=os.environ["AI_API_KEY"])
result = client.embeddings.create(
    model=os.environ["AI_EMBEDDING_MODEL"],
    input=["How much do I have to pay?", "The monthly fee is paid at the office."],
)
vectors = [item.embedding for item in result.data]
print(len(vectors), "vectors, each with", len(vectors[0]), "numbers")

The embedding model is a different, smaller model than the chat model. Use the same embedding model for your documents and your questions, or the numbers will not be comparable.

Where Embeddings Are Used

• Semantic search: find answers by meaning
• RAG: find the right documents to give a chatbot (Part 2)
• Recommendations: "students who liked this course also liked..."
• Grouping: cluster thousands of customer messages by topic
• Duplicate detection: find the same question asked in different words

Exercise