M11 Decoupling SQS & SNS & MQ (SNS (topic (access point and…
M11 Decoupling SQS & SNS & MQ
credit card verification
at least 1 delivery?
ocasionally can deliver > 1 copy
can be in diff order
message process exactly once?
in the same order as delivery
Have 3 buyer, bid with same price
So can use this as Tie breaker
Managed queue ?
no administration and little config
Visibility timeout ??
Max is 12hours (M: night)
during this timer, the message will be invisbile
so that other consumer will not poll it
Immediate after the essage is being consumed (received), the invisibiltiy timeout will start
Min is 0 seconds
!!!default is 30 second
Dead Letter queue ??
1.a queue of messages that could not be processed.
It receives messages after a maximum number of processing attempts (range is 1 to 1000) has been reached.
Messages can be sent to it and received from it like any other SQS queue.
Dead-letter queues are useful for debugging your application or messaging system because let udetermine why their processing doesn't succeed
shot polling (default)
Only query a
of servers when consumer poll the message
return immediately even when message queue is empty
can change to long polling ??
llowing SQS to
wait until a message is available
in a queue before sending a response
reduced cost as it reduce number of request to Make to SQS
within a single, highly-available AWS Region with multiple redundant Availability Zones, so that no single computer, network, or Availability Zone failure can make messages inaccessible. Messages can be sent and read simultaneously.
Server-side encryption (SSE) protects the contents of messages in Amazon SQS queues using keys managed in the AWS Key Management Service (AWS KMS). SSE encrypts messages as soon as Amazon SQS receives them. The messages are stored in encrypted form and Amazon SQS decrypts messages only when they are sent to an authorized consumer.
Developers can securely share Amazon SQS queues anonymously or with specific AWS accounts. Queue sharing can also be restricted by IP address and time-of-day.
store , process delete ?
responsibility of consumer to delete after process, if not will stay until 4 days (default), can change from 1 to 14 days
Means no notification when there is a message
Easily recover from fail step ?
Keep processing messages until the expiration of 4 days (by defualt)
minimizes the chances of your queue being blocked by messages that can’t be processed.
Use case slide
suitable application that
CAN not all process the same amount of work simultaneously.
Buffering batch operations:?
smooth out temporary volume spikes
without losing messages or increasing latency.
Request offloading: solve slow operations off of interactive request paths by enqueueing the request.
Each image requires resizing and encoding before it can be published.
The app runs on EC2 instances in an Auto Scaling group that is configured to handle your typical upload rates
An Auto Scaling group to manage EC2 instances for the purposes of processing messages from an SQS queue.
A custom metric to send to CloudWatch that measures the number of messages in the queue per EC2 instance in the Auto Scaling group.
A target tracking policy that configures your Auto Scaling group to scale based on the custom metric and a set target value. CloudWatch alarms invoke the scaling policy.
what happen if your upload rate change ?
when image is uploaded, the data of the image will be placed in the SQS
Suppose that you have a web app that lets users upload images and use them online.
1.Auto-scaling trigger based on SQS queue
draw digram before present
up to 256 KB of text
format can be String, Number, and Binary
handle perf by increase job instance ?
increase more consumers
increase thread per consumer
SERVICE TO SERVICE ??
a hotel booking system needs to cancel a booking and this process takes a long time.
Alternatively, you can put a message into a queue and have the same hotel booking system consume messages from that queue and perform asynchronous cancellations.
STATE CHANGE NOTIFICATION
For example, an inventory system may publish notifications when a certain item is low and needs ordering.
used by applicatino for API purpose
access point and channel
TO SEND OUT NEWS, LATER CLICK THE URl
Auto scaling send alert about changes
to rquest to install soft2ware or patches
Auto scaling hit target tracking
Example of business logic
MQ advanced feature
MQTT is a lightweight and widely adopted messaging protocol that is designed for constrained devices. AWS IoT support for MQTT is based on the MQTT v3.1.1 specification,
It offers queues and topics, durable and non-durable subscriptions, push-based and poll-based messaging, and filtering.