Please enable JavaScript.
Coggle requires JavaScript to display documents.
17.197: Order data into Kinesis - Coggle Diagram
17.197: Order data into Kinesis
Kinesis
Using a "Partition Key" value of the "truck_id"
Same key will always go to the same shard
SQS
No ordering
For SQS FIFO, if not use Group ID, messages are consumed in the order they are sent,
with only 1 consumer
You want to scale consumer number, but you want message to be "grouped" when they are related to each other
then you use a group ID (same as Partition key in Kinesis)
Example
Let assume 100 trucks, 5 kinesis shard, 1 SQS FIFO
Kinesis
20 trucks/shard
Max consumer in parallel is 5
Can receive up to 5MB/s of data
SQS FIFO
Have 100 group ID
Have up to 100 Consumers
Have up tp 300 message per second (or 3000 if using batching)
