Course Overview
Build a document question-answering system that runs locally on your computer using Python, Ollama, LangChain, and ChromaDB. This course guides you through setting up local generation and embedding models, extracting and cleaning text, building a vector database, composing a retrieval-augmented generation (RAG) pipeline, and evaluating and improving the system without relying on cloud LLM APIs.
You will also create a local Gradio document QA application with source display, incremental indexing, and follow-up question handling. The course emphasizes practical evaluation techniques to diagnose retrieval and generation failures and optimize the system effectively.
Key Takeaways
- Run local generation and embedding models with Ollama and call them from Python.
- Explain sentence embeddings and calculate cosine similarity using NumPy.
- Extract, clean, and split text while preserving context and source metadata.
- Build and update a persistent ChromaDB document index.
- Compose a retriever, prompt, local LLM, and parser with LangChain and LCEL.
- Build an evaluation set to distinguish retrieval failures from generation failures.
- Compare retrieval methods including BM25, hybrid retrieval, RRF, reranking, and prompt changes using recorded results.
- Create a local Gradio document QA app with source display, updates, and follow-up questions.
Prerequisites
- Basic Python knowledge: variables, functions, imports, and running scripts.
- Comfort with using a terminal and installing development tools.
- Python 3.12, Ollama, and a computer suitable for local models (16 GB RAM or more recommended).
- At least 15 GB of free disk space for models and documents.
- An internet connection for initial package and model downloads (no paid cloud LLM API key required).
Target Learners
- Python developers building document search or question-answering prototypes.
- Engineers who want to evaluate and debug RAG systems beyond basic demos.
- Developers exploring local models for workflows requiring careful document control.
- Learners with basic Python knowledge seeking a practical introduction to embeddings, vector databases, and grounded generation.
1.1.What You'll Build: A Local Document QA System 2:22
1.2.How RAG Works in Three Steps 1:58
1.3.RAG or Fine-Tuning: Which Do You Need? 1:48
1.4.Why Run RAG Locally? 1:48
1.5.Course Roadmap, Requirements and Troubleshooting 2:09
2.1.Phase 0: Environment Setup Goals 1:01
2.2.Install Ollama and Check the Local Service 1:29
2.3.Download and Run Qwen2.5 1:34
2.4.Download the bge-m3 Embedding Model 1:14
2.5.Create a Python Environment and Install Packages 1:24
2.6.Select the Right Jupyter Kernel in VS Code 1:25
2.7.Smoke Test: Generation and Embeddings 1:15
3.1.Phase 1: Understand Why Retrieval Works 0:48
3.2.What Does It Mean to Turn Text into a Vector? 1:26
3.3.Generate and Inspect an Embedding 1:20
3.4.Calculate Cosine Similarity with NumPy 1:10
3.5.Rank by Similarity: A Minimal Search Engine 1:29
3.6.Compare Embedding Models on English Examples 2:11
3.7.Where Embeddings Can Fail 1:48
3.8.Phase 1 Review: Making Retrieval Measurable 0:50
4.1.Phase 2: Build a Document Index 0:47
4.2.Course Data and Source Licenses 1:47
4.3.Extract Text from PDFs and HTML 1:37
4.4.Clean Headers, Footers and Extra Blank Lines 1:27
4.5.Design Chunks around Meaning 1:10
4.6.Use RecursiveCharacterTextSplitter 1:25
4.7.Measure the Effect of Chunk Size 1:49
4.8.Design Metadata for Source Attribution 1:30
4.9.Store and Query Chunks in ChromaDB 1:42
5.1.Phase 3: Connect Retrieval to Generation 0:50
5.2.Where LangChain Helps and When to Skip It 1:13
5.3.Create a Retriever 1:24
5.4.Design a Grounded Prompt Template 1:35
5.5.Connect the Pipeline with LCEL 1:35
5.6.Run Your First RAG Pipeline 1:15
5.7.Display Sources with Every Answer 1:36
5.8.Inspect the Limits of Naive RAG 1:46
6.1.Phase 4: Evaluate Before You Optimize 1:20
6.2.Build a 30-Question Evaluation Set 2:23
6.3.Recall, MRR and nDCG Explained 2:14
6.4.Measure an English-Language Baseline 2:06
6.5.Diagnose Retrieval versus Generation Failures 1:46
6.6.Improve Prompts with Controlled Comparisons 1:58
6.7.BM25, Hybrid Retrieval and RRF 1:47
6.8.Reranking: Measure Both Retrieval and Answers 1:36
6.9.Compare the Experiments and Choose a Configuration 1:52
7.1.Phase 5: the Finished Application 1:19
7.2.Define What the Application Should Answer 1:53
7.3.Load the Fictional Corporate Policies 1:40
7.4.Turn Ingestion into a Reusable Script 1:39
7.5.Incremental Updates and Reindexing 1:49
7.6.Finalize the Prompt and Run Regression Checks 1:32
7.7.Design Refusals and Test Score Thresholds 2:03
7.8.Add Conversation History and Follow-Up Questions 1:42
7.9.Build a Minimal Gradio Interface 1:55
7.10.Try Your Own Documents 1:41
8.1.When Should You Rebuild the Index? 1:31
8.2.Options When Your Workload Grows 1:40
8.3.A Map of Advanced RAG Techniques 1:59
8.4.Combine RAG with LoRA Fine-Tuning 1:21
8.5.Choose Hardware and Models for Your Task 1:39
8.6.Five Projects to Build Next 2:13
Access Files No pi required.
Free Course files 1 file package
- Full Pack
Discussions are closed
Comments are currently disabled for this course.
