AI-103: Developing AI Apps and Agents on Azure

In this AI-103: Developing AI Apps and Agents on Azure course, you'll learn:

Introduction to Developing AI Apps and Agents on Azure

• Overview of Microsoft Foundry as the unified platform for AI development and governance
• Audience profile and responsibilities of an Azure AI engineer
• Overview of the five exam skill domains and how they map to real-world solution design
• Course logistics and hands-on lab environment setup

Plan and Manage an Azure AI Solution 

• Choosing the appropriate Foundry services for generative AI and agents
  • Selecting an appropriate model for each task, including LLMs, small language models, multimodal models, and Foundry Tools
  • Choosing Foundry services for generative tasks, grounding, vector search, agent workflows, and multimodal processing
  • Choosing an appropriate retrieval and indexing method
  • Selecting memory, tool, and knowledge integration services for agent solutions
• Setting up AI solutions in Foundry
  • Designing Azure infrastructure for AI apps and agent-based solutions
  • Choosing appropriate deployment options
  • Configuring model and agent deployments
  • Integrating Foundry projects with CI/CD pipelines
• Managing, monitoring, and securing AI systems
  • Managing quotas, scaling, rate limits, and cost footprints for model and agent workloads
  • Monitoring model performance, drift, safety events, and grounding quality
  • Monitoring data ingestion quality, search index health, and relevance performance
  • Configuring security, including managed identity, private networking, keyless credentials, and role policies
• Implementing responsible AI across generative and agentic systems
  • Configuring safety filters, guardrails, risk detection, and content moderation
  • Applying responsible AI instrumentation, including evaluators, safety evaluations, and explanation tooling
  • Implementing auditing through trace logging, provenance metadata, and approval workflows
  • Governing agent behavior with oversight modes, constraints, and tool-access controls

Implement Generative AI and Agentic Solutions

• Building generative applications by using Foundry
  • Deploying and consuming LLMs, small models, code models, and multimodal models
  • Implementing retrieval-augmented generation (RAG) in an application
  • Designing workflows, tool-augmented flows, and multistep reasoning pipelines
  • Evaluating models and apps, including detecting fabrications, relevance, quality, and safety
  • Integrating generative workflows into applications using Foundry SDKs and connectors
  • Configuring an application to connect to a Foundry project
• Building agents by using Foundry
  • Defining agent roles, goals, conversation-tracking approach, and tool schemas
  • Building agents that integrate retrieval, function-calling, and conversation memory
  • Integrating agent tools, including APIs, knowledge stores, search, content understanding, and custom functions
  • Implementing orchestrated multi-agent solutions
  • Building autonomous or semiautonomous workflows with safeguards and approval flow controls
  • Integrating monitoring into deployed agents, evaluating agent behavior, and performing error analysis
• Optimizing and operationalizing generative AI systems
  • Tuning generation behavior through prompt engineering and model parameter adjustment
  • Implementing model reflection, chain-of-thought evaluations, and self-critique loops
  • Setting up observability with tracing, token analytics, safety signals, and latency breakdowns
  • Orchestrating multiple models, flows, or hybrid LLM and rules engines

Implement Computer Vision Solutions

• Designing and implementing image- and video-generation solutions
  • Generating images from text prompts and reference media
  • Generating videos from text prompts and reference media
  • Configuring image-editing workflows, including inpainting, mask-based edits, and prompt-driven modifications
  • Implementing workflows to edit generated videos
  • Selecting and applying platform generation and editing controls
• Designing and implementing multimodal understanding workflows
  • Building a solution that analyzes visual context using multimodal models
  • Configuring concise or detailed captions for single or multiple images
  • Implementing visual question-answering grounded in visual evidence
  • Configuring alt-text and extended image descriptions aligned to accessibility guidelines
  • Implementing visual understanding using Azure Content Understanding in Foundry Tools
  • Implementing video analysis workflows to process and interpret video segments
  • Configuring single-task and pro-mode Content Understanding pipelines
  • Implementing solutions that identify objects, components, or regions within images or video
• Implementing responsible AI for multimodal content
  • Implementing filters to classify unsafe or disallowed visual content
  • Detecting and mitigating indirect prompt injection via embedded text in images
  • Enforcing visual policy rules, such as watermarks, prohibited symbols, brand usage, and inappropriate content detection

Implement Text Analysis Solutions

• Applying language model text analysis
  • Extracting entities, topics, summaries, and structured JSON outputs using generative prompting and Foundry Tools
  • Configuring detection of sentiment, tone, safety issues, and sensitive content
  • Building solutions that translate text using Azure Translator in Foundry Tools or LLM-powered translation flows
  • Customizing language model outputs for domain tasks, such as compliance summarization and domain extraction
• Implementing speech solutions
  • Implementing workflows to convert speech to text and text to speech for agentic interactions
  • Integrating speech as an agent modality, including custom speech models
  • Enabling multimodal reasoning from audio inputs
  • Translating speech into other languages using language models and Foundry Tools

Implement Information Extraction Solutions 

• Building retrieval and grounding pipelines
  • Ingesting and indexing content, such as documents, images, audio, and video
  • Configuring semantic search, hybrid search, and vector search for grounding
  • Implementing enrichment using custom or built-in skills for text, images, and layout
  • Configuring RAG ingestion flow, including documents and OCR
  • Connecting retrieval pipelines directly to workflows and agent tools
• Extracting content from documents
  • Extracting information using multimodal pipelines that combine OCR, layout analysis, and field extraction
  • Producing clean, grounded representations for use with agents and RAG using Content Understanding
  • Implementing analyzers for structured or markdown outputs for downstream reasoning using Content Understanding