Categorizing Uploaded Data Using AWS Step Functions

While some resources have been provided, you will need to complete the following steps to finish configuring the environment.

1. Create an IAM role to allow Step Functions to start Lambda functions.
2. Create a Step Function. Use a default configuration for now, which will be properly set up later. You need the ARN of the Step Function.
3. In the run-step-function-lambda, set the STATEMACHINEARN environment variable to the ARN of the Step Function you just created.
4. Create an S3 Event Notification that will call the run-step-function-lambda.
5. Restrict the notification to mp3 data created in the upload folder.

Implement the following logic as a Step Function flow. Refer to the lab instructions to find resources already provided that will do some of these things for you. You can also reference the lab diagram for an example architecture.

1. Create a Transcribe job to translate the audio uploaded to S3 into text.
2. The Transcribe job can take a few minutes to run, so wait for it to complete.
3. After 30 seconds, check the status of the Transcribe job to see if it has completed.
4. If the Transcribe job has not completed, wait another 30 seconds for it to complete.
5. If the Transcribe job has failed, stop the pipeline with an error.
6. Once the Transcribe job has completed successfully, move the audio and transcript into categories based on the presence of keywords in the transcript.

The Lambda functions that are used in the Step Function flow have been partially written, but there is a lot still to do. Complete all of the TODO lines.

These functions are using the IAM role transcribe-audio-lambda-role, which gives them access to S3, Transcribe, and CloudWatch.

1. Finish writing transcribe-audio-lambda.
   • Get the state from the function's input.
   • Construct the parameters needed for the Transcribe job.
   • Add the transcript's file key and the Transcribe job name into the state.
2. Finish writing transcribe-status-lambda.
   • Retrieve the Transcribe job name from the state.
   • Add the job's status into the state.
3. Finish writing categorize-data-lambda.
   • Retrieve the state information. You've seen it done twice. Do you remember how to now?
   • Extract pieces of the state for further processing.
   • Determine the output location. Read the preceding code blocks.
   • Move the files to their proper destinations.
   • Note that this function relies on the environment variable KEYWORDS, which is a comma-separated list of words that are sensitive. If you are using the provided audio file, you can set this to important if you want the audio to match, or boring if you don't want it to match.

Helpful Resources:

• Boto3 Transcribe service
• Boto3 S3 service
• Python uuid module
• Python datetime module

With the previous steps complete, our serverless app is ready to do work! Let's test it out.

1. Create an upload folder in the provided S3 bucket.
2. Upload a test audio file to the upload folder.
3. View the Input and Output of the steps in the Step Function as the state machine executes.
4. Once your Step Function has completed, view the upload folder of the S3 bucket to confirm the file is no longer there.
5. View the folder you categorized your audio into.