Vector Embeddings and Semantic Search: Building RAG Pipelines from Scratch

Traditional databases search for exact string matches using SQL queries like WHERE title LIKE '%payment%'. But what happens if a user searches for "how do I send money to my supplier?"

Keyword search fails because the word "payment" is never explicitly stated.

To bridge this gap, modern AI systems use Vector Embeddings and Retrieval-Augmented Generation (RAG). At Kone AI, we teach engineers how to build semantic retrieval engines from first principles.

🧭 1. What is an Embedding?

An embedding model (like text-embedding-3-small) maps arbitrary text into a dense vector of numbers in high-dimensional space (e.g. 1,536 dimensions).

In this geometric space, texts with similar meanings sit close to each other:

"car" and "automobile" will have almost identical vectors.
"king" - "man" + "woman" \approx "queen"

The Math: Cosine Similarity

To measure how relevant two documents are, we calculate the cosine of the angle between their vectors $\mathbf{A}$ and $\mathbf{B}$:

$$\text{Cosine Similarity} = \frac{\mathbf{A} \cdot \mathbf{B}}{\|\mathbf{A}\| \|\mathbf{B}\|} = \frac{\sum_{i=1}^{n} A_i B_i}{\sqrt{\sum_{i=1}^{n} A_i^2} \sqrt{\sum_{i=1}^{n} B_i^2}}$$

1.0: Identical semantic direction
0.0: Completely orthogonal (unrelated)
-1.0: Diametrically opposite

🛠️ 2. The 4-Stage RAG Pipeline

[ Raw Documents ] ──> 1. Chunking ──> 2. Embedding Model ──> 3. Vector Database (pgvector/Pinecone)
                                                                             │
[ User Query ] ───> Embed Query ───> Cosine Distance Match <────────────────┘
                                              │
                                              ▼
                                    Top 3 Relevant Chunks + Prompt
                                              │
                                              ▼
                                     [ LLM Generation ] ──> Accurate Answer

💻 3. Building a Pure TypeScript Similarity Search

Here is how you compute vector similarity in TypeScript without external dependencies:

export function cosineSimilarity(vecA: number[], vecB: number[]): number {
  if (vecA.length !== vecB.length) {
    throw new Error('Vector dimensions must match');
  }

  let dotProduct = 0;
  let normA = 0;
  let normB = 0;

  for (let i = 0; i < vecA.length; i++) {
    dotProduct += vecA[i] * vecB[i];
    normA += vecA[i] * vecA[i];
    normB += vecB[i] * vecB[i];
  }

  const denominator = Math.sqrt(normA) * Math.sqrt(normB);
  if (denominator === 0) return 0;

  return dotProduct / denominator;
}

Dive deep into PyTorch, RAG architectures, and vector search in our Neural Architectures & Vector Search Track.

Vector Embeddings and Semantic Search: Building RAG Pipelines from Scratch