Featured resource
Tech Upskilling Playbook 2025
Tech Upskilling Playbook

Build future-ready tech teams and hit key business milestones with seven proven plays from industry leaders.

Learn more
  • Course
    • Libraries: If you want this course, consider one of these libraries.
    • AI

Evaluating and Optimizing LLM Agents

Learn to evaluate and optimize LLM agents using tools like G-Eval, DeepEval, and LangSmith. Apply metrics, build custom tests, and tune quality, cost, and latency for real-world performance and reliability

Brian Letort - Pluralsight course - Evaluating and Optimizing LLM Agents
by Brian Letort

What you'll learn

This course is designed for AI engineers, developers, and data scientists building intelligent agents who must ensure those agents produce accurate, relevant, and efficient responses—especially in complex enterprise environments. In this course, Evaluating and Optimizing LLM Agents, you’ll gain the skills needed to assess and enhance agent performance in real-world environments. First, you’ll explore core evaluation metrics like answer relevancy, hallucination rate, and contextual fit, and apply them using tools such as G-Eval and DeepEval. Next, you’ll create domain-specific test suites with open-rag-eval and build dashboards with LangSmith to monitor performance across cost, latency, and quality. Finally, you’ll learn how to apply these strategies across various architectures, including RAG agents, multi-agent systems, and chat-based tools. When you’re finished with this course, you’ll have a practical, repeatable framework for evaluating and optimizing LLM agents at scale.

Table of contents

About the author

Brian Letort - Pluralsight course - Evaluating and Optimizing LLM Agents
Brian Letort

Dr. Daniel “Brian” Letort is a 22+ year veteran of Information Technology. During a 21-year tenure at Northrop Grumman, Brian held various roles across software engineering, systems engineering, Chief Applications Architect, Chief Data Scientist, and Chief Enterprise Architect. Brian held the NG Fellow title for six years and Technical Fellow title for four years prior. In 2022, Brian joined Digital Realty as the Chief Architect - Product and Artificial Intelligence. Aside from working at Digital Realty, Brian has 12+ years experience in teaching Data Science and Computer Science classes as an adjunct professor. Brian has authored two books and holds two patents.

More Courses by Brian