What you'll learn

Latency, error rates, and probes all stay green when a prompt change quietly makes an assistant's answers worse, so no dashboard can tell you whether a proposed prompt is safe to ship. In this course, Demo: Gate a Prompt Change on Evaluation Evidence in Azure, you’ll gain the ability to decide whether a prompt change should ship by comparing it to the current prompt on the same evaluation set. First, you’ll explore how to run a fixed evaluation set against the current prompt and the proposed one in Microsoft Foundry, changing nothing but the system prompt between the two runs. Next, you’ll discover how to read the two results side by side to find the quality loss that operational metrics never show, and how to judge whether a difference on a small set is big enough to act on. Finally, you’ll learn how to make and defend a ship, block, or look-closer call and record the evidence a reviewer would need to accept it. When you’re finished with this course, you’ll have the skills and knowledge of evaluation-based prompt gating needed to put your own next prompt change through the same gate before it reaches production.