This course runs for 2h 24m. It is taught by Balaji S, published by Udemy, and was released on 2026-09-20. The course taught in en-US and includes exercise files.
Course Overview
This course provides QA professionals with a practical framework to test AI systems, including LLMs, Retrieval-Augmented Generation (RAG), and AI agents. It covers evaluation strategies, security checks, CI/CD integration, and hands-on experience with a runnable QA Shop Assistant sample app to investigate real failure modes.
Students will learn to design evaluation strategies, validate AI outputs for correctness, safety, and consistency, test agent behavior and tool use, and implement security and safety evaluations. The course also covers running evaluations in CI/CD pipelines and monitoring in production.
Key Takeaways
Design evaluation strategies for LLM, RAG, and agentic systems using deterministic, model-graded, and human-review checks.
Test AI outputs for correctness, safety, grounding, relevance, consistency, and regression risk.
Validate RAG retrieval, citations, context quality, and failure behavior through practical QA scenarios.
Test agent tool use, permissions, traces, security controls, and CI/CD quality gates using a working sample app.
Prerequisites
Basic software-testing knowledge is helpful. No prior AI or LLM automation experience is required. A computer is needed for the local lab.
Target Learners
Manual testers, QA engineers, SDETs, test leads, and QE architects who want to expand into AI and agentic-system quality engineering.
