Similarity from scratch

Words as vectors

To find related chunks, turn each text into a vector (a list of numbers) and compare vectors.

Real systems use embeddings from a model, which capture meaning ("refund" is close to "money back"). The idea is the same with plain word counts, which you can build in ten lines: each word is a direction, and its count is how far you go.

Cosine similarity measures the angle between two vectors: 1 means same direction, 0 means nothing in common. Long and short texts compare fairly because length cancels out.

How similar?

import math

def vector(text):
    counts = {}
    for word in text.lower().split():
        counts[word] = counts.get(word, 0) + 1
    return counts

def cosine(a, b):
    dot = sum(a[w] * b.get(w, 0) for w in a)
    size = math.sqrt(sum(v * v for v in a.values())) * math.sqrt(sum(v * v for v in b.values()))
    return dot / size if size else 0.0

q = vector("refund my order")
d = vector("how to get a refund for an order")
print(round(cosine(q, d), 3))

What does this print?

• 0.408
• 0.5
• 0.333

Exercise: Find the best chunks

Input: k, then the question, then n, then n chunks as id|text.

Score each chunk with cosine similarity of lower-cased word counts (ignore punctuation .,?!). Print the top k as id score (score to 3 decimals), best first; ties keep their input order.