Advanced Context Engineering: Systems, Evaluation & Research

Course Overview

This course teaches how to build, measure, and test context systems for text, knowledge, and agents, enabling learners to verify context, manage long context, build memory with provenance, and evaluate systems systematically. It covers practical techniques including text extraction, retrieval methods comparison, context assembly, memory management, long task handling, and research paper evaluation.

Key Takeaways

• Extract and assemble verifiable context from documents with traceable claims.
• Compare and select retrieval methods such as BM25, embeddings, rank fusion, and reranking based on measured evidence.
• Chain dependent searches and audit context graphs with permission checks.
• Manage long context using budgeted selection, shortening, handoffs, and caching strategies.
• Build and test memory systems with provenance, validity dates, and lifecycle rules.
• Develop local MCP servers with typed inputs, pagination, and failure handling.
• Run and resume long tasks with explicit state and checkpoints, comparing agent performance.
• Detect hidden instructions, enforce permissions, and test for leaks and over-blocking in code.
• Systematically evaluate systems using checkers, hand labels, regression detection, and ablation studies.
• Read and test research papers against fair baselines and complete systems on new domains with sealed tests.

Prerequisites

The course builds on Context Engineering, Part I, and expects learners to write code or be ready to learn programming as they go. Setup requires a Linux shell, Python 3.12, Codex CLI with an account, and a paid embedding service for some lessons.

Target Learners

This course is designed for developers, researchers, and practitioners interested in advanced context engineering, AI agents, and system evaluation who have some programming experience or are willing to learn programming during the course.

Final Project

The course includes a final project involving a fictional warehouse handbook, culminating in a sealed test that is opened once, applying the skills learned to build a full context system.