Featured resource
2025 Tech Upskilling Playbook
Tech Upskilling Playbook

Build future-ready tech teams and hit key business milestones with seven proven plays from industry leaders.

Check it out
  • Lab
    • Libraries: If you want this lab, consider one of these libraries.
    • Cloud
    • Data
Google Cloud Platform icon
Labs

Utilizing Write Sharding to Optimize Data Ingestion

In this lab, we investigate and improve a DynamoDB table loading script that is losing data by modifying the data item partition key to shard the table partitions.

Google Cloud Platform icon
Lab platform
Lab Info
Level
Advanced
Last updated
Aug 23, 2025
Duration
45m

Contact sales

By filling out this form and clicking submit, you acknowledge our privacy policy.
Table of Contents
  1. Challenge

    Investigate Provided Instance and Data

    Log in to the provided EC2 instance with the credentials provided in the lab information. Look at the data being written to the table with the following command:

    cat dataload/bin_meta_10  | jq
    
  2. Challenge

    (OPTIONAL) Run Unmodified `loadtable.py`

    Run loadtable.py.py with the following command, and observe the results in the DynamoDB web console:

    python3 loadtable.py > load.log &
    

    (If you do this step, re-create the AmazonBins table with a partition key of Partition, which is a string, and a sort key of bin, which is also a string.)

  3. Challenge

    Modify `loadtable.py`

    Edit the transform function in loadtable.py to modify the Partition key for each item to create multiple partitions in the table. This can be accomplished by:

    • Generating random characters
    • Assigning alphanumeric partitions to records
    • Using some value from the existing data to increase the cardinality of the values stored in the Partition key
  4. Challenge

    Run Modified `loadtable.py`

    Run your modified version of loadtable.py:

    python3 loadtable.py > load2.log &
    
  5. Challenge

    Observe Results

    In the DynamoDB console in the Metrics tab, observe the write capacity unit usage, throttled write request, and throttled write events. WCU usage should be around 2000, and both throttle metrics should be zero.

About the author

Pluralsight Skills gives leaders confidence they have the skills needed to execute technology strategy. Technology teams can benchmark expertise across roles, speed up release cycles and build reliable, secure products. By leveraging our expert content, skill assessments and one-of-a-kind analytics, keep up with the pace of change, put the right people on the right projects and boost productivity. It's the most effective path to developing tech skills at scale.

Real skill practice before real-world application

Hands-on Labs are real environments created by industry experts to help you learn. These environments help you gain knowledge and experience, practice without compromising your system, test without risk, destroy without fear, and let you learn from your mistakes. Hands-on Labs: practice your skills before delivering in the real world.

Learn by doing

Engage hands-on with the tools and technologies you’re learning. You pick the skill, we provide the credentials and environment.

Follow your guide

All labs have detailed instructions and objectives, guiding you through the learning process and ensuring you understand every step.

Turn time into mastery

On average, you retain 75% more of your learning if you take time to practice. Hands-on labs set you up for success to make those skills stick.

Get started with Pluralsight