AI Architecture: Designing LLM Systems for the Enterprise

Course Overview

This course teaches how to design the architecture of AI systems within a real enterprise context, focusing on safety, affordability, and compliance for AI deployments. Using a case study of a fictional European insurer, learners explore AI architecture fundamentals, model gateways, retrieval systems respecting access rights, agent design with budget and limits, security measures against prompt injection, audit trails, EU AI Act classification, and evaluation for release readiness. The course includes hands-on labs to create architecture documents and diagrams without requiring coding skills.

Key Takeaways

• Choose appropriate AI architecture patterns from single model calls to multi-agent systems and document them with C4 diagrams.
• Assess AI use case feasibility and record decisions in architecture decision records.
• Design model gateways controlling model usage, data processing, and cost management.
• Implement Retrieval-Augmented Generation (RAG) respecting document versions and access rights, and decide between prompting or fine-tuning.
• Design AI agents with clear operational limits and budgets, and evaluate multi-agent coordination protocols.
• Secure and govern LLM systems with guardrails against prompt injection, data residency compliance, EU AI Act classification, and audit trails.
• Evaluate AI systems using baselines and release gates, and conduct comprehensive architecture reviews before deployment.

Prerequisites

• Comfortable reading technical documents such as logs, configuration files, and JSON records.
• No prior AI or machine learning background required.
• No need for GPU, Docker, virtual machines, IDEs, Jupyter, or database servers; any laptop and operating system suffice.

Target Learners

• Software, solution, and enterprise architects assigned AI initiatives.
• Senior engineers and tech leads making architectural decisions for AI projects.
• Individuals managing systems integrating AI components.
• Architects interested in governance, security, and evaluation aspects of AI architecture.