Featured resource
2026 Tech Forecast
2026 Tech Forecast

1,500+ tech insiders, business leaders, and Pluralsight Authors share their predictions on what’s shifting fastest and how to stay ahead.

Download the forecast
  • Course

Evaluating and Optimizing LLM Agents

This course will teach you how to improve answer quality, latency, reliability, and cost while building a continuous production improvement loop.

Beginner
2h

Created by Brian Letort

Last Updated Nov 20, 2026

Course Thumbnail
  • Course

Evaluating and Optimizing LLM Agents

This course will teach you how to improve answer quality, latency, reliability, and cost while building a continuous production improvement loop.

Beginner
2h

Created by Brian Letort

Last Updated Nov 20, 2026

Get started today

Access this course and other top-rated tech content with one of our business plans.

Try this course for free

Access this course and other top-rated tech content with one of our individual plans.

This course is included in the libraries shown below:

  • AI
What you'll learn

A knowledge agent can perform well on prepared questions and still fail when real users bring new terminology, incomplete requests, or changing information. In this course, Evaluating and Optimizing LLM Agents, you'll gain the ability to evaluate and improve the complete agent workflow. First, you'll build task-specific benchmarks and assess answers, citations, tool choices, and execution paths using automated checks, model judges, and human review. Next, you'll improve retrieval, prompts, planning, memory, and model routing while measuring quality, latency, reliability, and cost. Finally, you'll trace production behavior, turning reviewed user feedback into regression tests, verified fixes, and controlled releases. When you're finished, you'll be able to establish a repeatable evaluation and improvement process for an enterprise knowledge agent.

Evaluating and Optimizing LLM Agents
Beginner
2h
Table of contents

About the author
Brian Letort - Pluralsight course - Evaluating and Optimizing LLM Agents
Brian Letort
18 courses 3.3 author rating 40 ratings

Dr. Daniel “Brian” Letort is a 22+ year veteran of Information Technology. During a 21-year tenure at Northrop Grumman, Brian held various roles across software engineering, systems engineering, Chief Applications Architect, Chief Data Scientist, and Chief Enterprise Architect. Brian held the NG Fellow title for six years and Technical Fellow title for four years prior. In 2022, Brian joined Digital Realty as the Chief Architect - Product and Artificial Intelligence. Aside from working at Digital Realty, Brian has 12+ years experience in teaching Data Science and Computer Science classes as an adjunct professor. Brian has authored two books and holds two patents.

2025 Forrester Wave™ names Pluralsight as a Leader among tech skills dev platforms

See how our offering and strategy stack up.

forrester wave report