Please enable JavaScript.
Coggle requires JavaScript to display documents.
DynamoDB - Part 1 - Coggle Diagram
DynamoDB - Part 1
-
-
Capacities R/W
WCU
- One WCU represents one write per second for an item up to 1KB in size
Examples
- We write 10 items per second, with item size 2KB. We will need to have 20 WCUs
- Calculation 10 * (2 / 1) = 20 WCUs
- We write 6 items per second, with item size 4.5KB (you need to rounded to upper)
- Calculation 6 * (5/1) = 30 WCUs
- We write 120 items per minute, with item size 2KB
- Calculation (120/60) * (2 KB / 1 KB) = 4 WCUs
RCU
- 1 RCU represents one Stronly Consistent Read per second for an item up to 4 Kb
- 2 RCU reperesents Eventually Consistent Read per second for an item up to 4 Kb
- If are larger than 4KB more RCUs are consumed
Examples
- 10 Strongly consistent reads per second, with item size 4Kb
- Calculation 10 * (4 KB / 4 KB) = 10 RCUs
- 16 Eventually Consistent reads per second, with item size 12KB
- Calculation (16 / 2) * (12 / 4) = 24 RCUs
- 10 Strongly Consistent Reads per seconds, with item size 6KB
- Calculation 10 * (8 / 4) = 20 RCUs
DynamoDB - NoSQL
- It's NoSQL database
- The database are distributed
- MongoDB, DynamoDB
- Not supports query joins
- NoSQL doens't have aggregations such as SUM, AVG...
- NoSQL databases scale horizontally
Amazon DynamoDB
- Fully managed
- Highly available with replication across multiple AZs
- Scales to massive workloads
- Millions of requests per seconds, trillions of rows, 100s of TB of storage
- Fast and consistent (low latency)
- IAM for security
- Enables event driven programming with DynamoDB Streams
- Low cost and auto-scaling capabilities
- Standard and infrequent access (IA) Table Class
DynamoDB Basics
- There are tables
- Each table has a primary key
- Infinite number of items (rows)
- Each item has attributes (can be null)
- Maximum size 400KB
- Scalar Types String, Number, Binary, Boolean, Null
- Document types: List, Map
- SetTypes: String Set, Number Set, Binary Set
Strongly Consistent Read vs Eventually Consistent Read Eventually Consistent Read (default)
- If we read just after a write, it's possible we will get some stale date because of replication
Strongly Consistent Rad
- If we read just after a write, we will get the correct data
- You need to set ConsistentRead parameter to True in API Calls (GetItem, BatchGetItem, Query, Scan)
- Consumes twice the RCU and it's little slowly
Partitions Internal
- Data is storage in partitions
- Partitions keys go thought a hashing algorithm to know to which partition they go to
Compute the number of partitions
- (RCUs / 3000) + (WCUs / 1000) (by capacity)
- (total size / 10 Gb) (by size)
- ceil(max((by capacity), (bysize)
- WCU and RCU are spread evenly across partitions
Throttling
- ProvisionedThorughtputExceededException
Reasons
- Hot keys - one partition key is being used read too many times
- Hot Partitions -
- Very large items
Solutions
- exponencial backoff
- Distribuited partition keys as much as possible
- If RCU issue, we can use DynamoDB Accelerator (DAX)
R/W Capacity Modes - On Demand
- No errors
- No throttle
- More expensive
- Unlimited RCU and WCU
- Pay for use
Writting Data
- PutItem
- create or replace the item
- UpdateItem
- edit an existent item or adds if not exists (only execute it in some fields
- Can be used to implement Atomic Counters
Conditions Writes
- Accept a write/update/delete only if conditions are met, otherwise return error
- Helps with concurrent accessto items
Query
- KeyConditionExpression
- PK
- SortKey
Filter expression
- Add filtering after the query operation
- Use only with non-key attributes (does not allow hash + range)
Return
- Number of items speciffied in limit
- Or up to 1MB data
- Ability to do the pagination
- You can query table, a local secondary index, or global secondary index
Reading Data
- Read PK
- PK can be hash or hash+range
- Eventually consistent read (default)
- Strongly Consistent Reads (more RCU)
- ProjectionExpression can be specified to retrieve only certain attributes
DynamoDB - Condition Writes
- For PutItem, UpdateItem, DeleteItem, BatchWriteItem
- You can define some condition
- attributes_exists, attributes_not_exists, attributes_type, contains, begins_with, ProductCategory in, Price between, size
- Filter expression filters the results of read queries, while condition expressions are for write operations
Example
aws dynamodb update-item --table-name ProductCatalog --key '{"id": {"N": "456"}}' --update-expression "SET Price = Price -:discount" --condition-expression "Price>=:limit" --expression-attributes-values file://values.json
DynamoDB - Local Secondary Index (LSI)
- Alternative Sort Key for your table (same Partition Key as that of base table)
- Scalar attribute
- Up to 5 Local Secondary Indexes per table
- Must be defined at table creation time
- Attributes Projections - can contain some or all the attributes of the base table (KEYS_ONLY, INCLUDE_ALL)
- Create LSI to filter using some specific field
DynamoDB - Global Secondary Index (GSI)
- Alternative PK (HASH + HAHS+RANGE) from the base table
- Speed up queries on non-keys attributes
- Attributes Projections - can contain some or all the attributes of the base table (KEYS_ONLY, INCLUDE_ALL)
- Must provision RCUs and WCU for the index
- Can be added/modified after table creation
DynamoDB - Indexes and ThrottlingGSI
- If the writes are thottled on the GSI, then the main table will be throttled
- Write your WCU properly
- Choose your GSI carefully
LSI
-Uses WCUs and RCUs of the main table
- No special throttling consideratons