Utilizing Write Sharding to Optimize Data Ingestion

Log in to the provided EC2 instance with the credentials provided in the lab information. Look at the data being written to the table with the following command:

cat dataload/bin_meta_10  | jq

Run loadtable.py.py with the following command, and observe the results in the DynamoDB web console:

python3 loadtable.py > load.log &

(If you do this step, re-create the AmazonBins table with a partition key of Partition, which is a string, and a sort key of bin, which is also a string.)

Edit the transform function in loadtable.py to modify the Partition key for each item to create multiple partitions in the table. This can be accomplished by:

• Generating random characters
• Assigning alphanumeric partitions to records
• Using some value from the existing data to increase the cardinality of the values stored in the Partition key

Run your modified version of loadtable.py:

python3 loadtable.py > load2.log &

In the DynamoDB console in the Metrics tab, observe the write capacity unit usage, throttled write request, and throttled write events. WCU usage should be around 2000, and both throttle metrics should be zero.