- Course
OpenAI Security and Moderations
An LLM can produce harmful output, and one moderation call is rarely enough to stop it. This course will teach you how to moderate text and images with OpenAI's Moderations API and wire those checks into a working application.
- Course
OpenAI Security and Moderations
An LLM can produce harmful output, and one moderation call is rarely enough to stop it. This course will teach you how to moderate text and images with OpenAI's Moderations API and wire those checks into a working application.
Get started today
Access this course and other top-rated tech content with one of our business plans.
Try this course for free
Access this course and other top-rated tech content with one of our individual plans.
This course is included in the libraries shown below:
- AI
What you'll learn
Most tutorials for building on the OpenAI API stop at a single moderation call and leave out the parts a real feature needs: thresholds, image and multi-modal checks, quarantine, audit logs, and a compliance story. In this course, OpenAI Security and Moderations, you'll gain the ability to build a content moderation layer around an OpenAI-based application. First, you'll explore how the Moderations API classifies text and images across its content categories, and how to read its scores and set per-category thresholds. Next, you'll discover how to assemble those checks into a working system, including input and output gates, image and combined text-image moderation, quarantine for human review, graduated responses, and audit logs. Finally, you'll learn how to defend the system against jailbreaks and prompt injection and map its controls to regional content regulations. When you're finished with this course, you'll have the skills and knowledge of content moderation needed to add a defensible safety layer to an application based on OpenAI’s API.