- Lab
-
Libraries: If you want this lab, consider one of these libraries.
- Cloud
Orchestrate an ML Data Intake Workflow
In this lab, you will design and implement an event-driven ML data intake workflow using AWS services. You will create a pipeline where incoming data uploaded to an Amazon S3 raw bucket automatically triggers a Step Functions workflow via AWS Lambda. The workflow will validate each file against basic intake requirements (such as structure and readability), and then route it to the appropriate outcome, either processing valid data for downstream ML use or handling invalid, empty, or corrupt files without failing the system. By the end of the lab, you will have built a failure-aware, production-style data ingestion pipeline that ensures reliable data flow and clear separation between successful processing and error handling.
Lab Info
Table of Contents
-
Challenge
Orchestrate an ML data intake workflow that validates and routes uploaded data
- Create three S3 buckets to represent different stages of the pipeline
- Set up S3 event notification on the Raw bucket
- Create a Lambda function to trigger a Step Function execution when a file is uploaded
-
Challenge
Implement data validation, data processing and error handling logic
- Create a Lambda function validate data
- Create a Lambda function process data
- Create a Lambda function handle error in data
-
Challenge
Build a Step Function to orchestrate the workflow and validate
- Build a Step Function to orchestrate the workflow
- Upload different types of files to the Raw bucket
- Validate final outputs
Real skill practice before real-world application
Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.
Learn by doing
Engage hands-on with the tools and technologies you’re learning. You pick the skill, we provide the credentials and environment.
Follow your guide
All labs have detailed instructions and objectives, guiding you through the learning process and ensuring you understand every step.
Turn time into mastery
On average, you retain 75% more of your learning if you take time to practice. Hands-on labs set you up for success to make those skills stick.