Part 1: Working with Text

Master embeddings, vector databases, and RAG—the foundation for any AI system that needs to understand and retrieve your own data.

What You'll Learn

This section covers the complete pipeline from raw text to intelligent retrieval. You'll start with the theory of embeddings and similarity metrics, run models locally with Ollama, store vectors in Qdrant, combine vector search with traditional databases, chunk documents for embedding, build a full RAG system, and visualize high-dimensional data.

Lesson 1: Embeddings

Understand vectors and embeddings as numeric representations of meaning. Learn why dimensionality matters for accuracy, memory, and speed. Compare cosine similarity, dot product, and Euclidean distance, then use SentenceTransformers to generate and compare embeddings in Python.

Lesson 2: Introducing Ollama

Run embedding models locally instead of depending on cloud APIs. Choose between models like bge-m3 and nomic-embed-text, understand trade-offs in latency and memory usage, and replace SentenceTransformers with an Ollama embedding call.

Lesson 3: Vector DBs Basics

Store and search embeddings efficiently with Qdrant, a purpose-built vector database. Learn its main building blocks—collections, points, payloads—and understand how payload filters and HNSW indexing enable fast similarity search at scale.

Lesson 4: Adding Traditional Storage

Understand why vector databases should only store embeddings and small metadata. Learn how traditional storage (SQLite) complements a vector database for large data, linking Qdrant semantic search results to full records via shared identifiers.

Lesson 5: Chunking & LangChain

Large documents must be split into smaller chunks before embedding. Use LangChain to load, split, and embed PDFs into Qdrant. Store both chunk text and embeddings directly in the vector database, assigning unique chunk IDs to avoid duplicates.

Lesson 6: Building a RAG

Put everything together into a Retrieval-Augmented Generation system. Combine embeddings, vector search, chunking, and an LLM to answer questions grounded in your own documents rather than the model's training data.

Lesson 7: Multi-Dimensional Data Visualization

Visualize high-dimensional embedding data in 2D using dimensionality reduction. Compare PCA, t-SNE, and UMAP, learn to read clusters and outliers from reduced plots, and understand the pitfalls of interpreting projected data.

Why This Matters

These seven lessons give you the complete text-to-retrieval pipeline. Without embeddings, your AI can't understand semantic meaning. Without vector databases, you can't search at scale. Without chunking, you can't process real documents. Without RAG, your models are limited to their training data.

Complete these lessons before moving to agentic AI in Part 2.