Please enable JavaScript.
Coggle requires JavaScript to display documents.
M11 Decoupling SQS & SNS & MQ (SNS (topic (access point and…
M11 Decoupling SQS & SNS & MQ
2 type
Standard
used
case
Upload media
credit card verification
at least 1 delivery?
ocasionally can deliver > 1 copy
can be in diff order
FIFO
message process exactly once?
in the same order as delivery
used case
Aution
Have 3 buyer, bid with same price
So can use this as Tie breaker
Managed queue ?
temporary repository
no administration and little config
SQS
config features
Visibility timeout ??
Max is 12hours (M: night)
during this timer, the message will be invisbile
so that other consumer will not poll it
Immediate after the essage is being consumed (received), the invisibiltiy timeout will start
Min is 0 seconds
!!!default is 30 second
Dead Letter queue ??
1.a queue of messages that could not be processed.
It receives messages after a maximum number of processing attempts (range is 1 to 1000) has been reached.
Messages can be sent to it and received from it like any other SQS queue.
Dead-letter queues are useful for debugging your application or messaging system because let udetermine why their processing doesn't succeed
No default
shot polling (default)
Only query a
subset
of servers when consumer poll the message
return immediately even when message queue is empty
can change to long polling ??
will queue
ALL
SQS servers
llowing SQS to
wait until a message is available
in a queue before sending a response
reduced cost as it reduce number of request to Make to SQS
feature
within a single, highly-available AWS Region with multiple redundant Availability Zones, so that no single computer, network, or Availability Zone failure can make messages inaccessible. Messages can be sent and read simultaneously.
Server-side encryption (SSE) protects the contents of messages in Amazon SQS queues using keys managed in the AWS Key Management Service (AWS KMS). SSE encrypts messages as soon as Amazon SQS receives them. The messages are stored in encrypted form and Amazon SQS decrypts messages only when they are sent to an authorized consumer.
Developers can securely share Amazon SQS queues anonymously or with specific AWS accounts. Queue sharing can also be restricted by IP address and time-of-day.
store , process delete ?
responsibility of consumer to delete after process, if not will stay until 4 days (default), can change from 1 to 14 days
Asyn ?
Means no notification when there is a message
Easily recover from fail step ?
Keep processing messages until the expiration of 4 days (by defualt)
minimizes the chances of your queue being blocked by messages that can’t be processed.
Use case slide
Work queues?
suitable application that
CAN not all process the same amount of work simultaneously.
Buffering batch operations:?
smooth out temporary volume spikes
without losing messages or increasing latency.
Request offloading: solve slow operations off of interactive request paths by enqueueing the request.
tm: mYSELF
Scaling trigger:
Each image requires resizing and encoding before it can be published.
The app runs on EC2 instances in an Auto Scaling group that is configured to handle your typical upload rates
dtdtdt
An Auto Scaling group to manage EC2 instances for the purposes of processing messages from an SQS queue.
A custom metric to send to CloudWatch that measures the number of messages in the queue per EC2 instance in the Auto Scaling group.
A target tracking policy that configures your Auto Scaling group to scale based on the custom metric and a set target value. CloudWatch alarms invoke the scaling policy.
what happen if your upload rate change ?
when image is uploaded, the data of the image will be placed in the SQS
Suppose that you have a web app that lets users upload images and use them online.
1.Auto-scaling trigger based on SQS queue
decouping
draw digram before present
Message ?
up to 256 KB of text
format can be String, Number, and Binary
handle perf by increase job instance ?
increase more consumers
increase thread per consumer
SERVICE TO SERVICE ??
a hotel booking system needs to cancel a booking and this process takes a long time.
Alternatively, you can put a message into a queue and have the same hotel booking system consume messages from that queue and perform asynchronous cancellations.
STATE CHANGE NOTIFICATION
For example, an inventory system may publish notifications when a certain item is low and needs ordering.
SNS
email-JSON
used by applicatino for API purpose
topic
access point and channel
SPECIFY POLICY
EMAIL
TO SEND OUT NEWS, LATER CLICK THE URl
alert
Auto scaling send alert about changes
Mobile push
to rquest to install soft2ware or patches
Used case
Auto scaling hit target tracking
SWF
Example of business logic
CRM
MQ advanced feature
MQTT is a lightweight and widely adopted messaging protocol that is designed for constrained devices. AWS IoT support for MQTT is based on the MQTT v3.1.1 specification,
feature rich
It offers queues and topics, durable and non-durable subscriptions, push-based and poll-based messaging, and filtering.
