Embeddings and cosine similarity

An embedding model turns a piece of text into a fixed-length vector of numbers, typically 256–3,072 of them. It's trained so that texts with similar meaning end up pointing in similar directions, even when they share no words:

"How do I reset my password?"    → [ 0.12, -0.48,  0.33, … ]
"I forgot my login credentials"  → [ 0.10, -0.45,  0.36, … ]   ← close
"Best pizza in Naples"           → [-0.51,  0.22, -0.07, … ]   ← far

#Measuring closeness

Cosine similarity is the cosine of the angle between two vectors: 1 means the same direction, 0 unrelated, −1 opposite.

import numpy as np

def cosine(a, b):
    return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))

Many embedding models return normalized vectors (length 1). Then cosine similarity equals the plain dot product, which is cheaper, and ranking by Euclidean distance gives the same order. Use whichever metric your model's documentation recommends, and configure your vector index to match.

Scores are only meaningful relative to each other, within one model. A cosine of 0.8 might be a strong match for one model and a weak one for another. Don't hard-code thresholds without testing them on your own data.

#What embeddings are good for

• Semantic search: find documents that answer a question phrased differently. This is the retrieval half of RAG.
• Deduplication and clustering: group near-identical tickets or reviews.
• Classification and routing: compare an input with labelled examples and pick the nearest.
• Recommendations: find items similar to ones a user liked.

#Limits

• Exact terms. Product codes, error IDs and names often match poorly. Combine embeddings with keyword search (hybrid search).
• Similar isn't the same as relevant. "How to cancel" and "how not to cancel" embed close together. Reranking helps.
• Vectors from different models, or different versions of one model, aren't comparable. Changing models means re-embedding everything.
• Long inputs get blurred into one vector. That's why documents are split up first (chunking).

Comparing a query with millions of vectors one by one is slow, so production systems use approximate nearest-neighbour indexes.