Please enable JavaScript.
Coggle requires JavaScript to display documents.
Scalability & High Availability - Coggle Diagram
Scalability & High Availability
Scalability
App adapts to load
Vertical scalability
Increase size of instance
Limited by hardware
Horizontal Scalability
Increase number of instances
Implies distributed system
High Availability
Linked to horizontal scaling
App runs in at least 2 AZs
Goal is to survive AZ loss
Load Balancing
Why use
Spread load
Single point of access (DNS)
Handle fail over of downstream instances
Health checks
SSL termination
Enforce cookie stickiness
High availability across AZs
Separate public vs. private traffic
Elastic Load Balancer
Health checks
Allows LB to know if traffic should be sent to instance
Done on port and route
Typically expects HTTP 200
Types
Classic Load Balancer (CLB)
TCP (L4) and HTTP/HTTPS (L7)
Health checks on TCP / HTTP
Fixed hostname
Deprecated
Application Load Balancer (ALB)
HTTP (L7)
Multiple apps across instances (target groups)
Multiple apps on same instances (containers)
Supports HTTP/2 & WebSocket
Supports redirects
Routing
Path
Hostname
Query strings
HTTP Headers
Can port map to dynamic port on ECS
Target groups
EC2 instances
ECS tasks
Lambda functions
Private IP addresses
Has fixed hostname
Source request detail headers
IP: X-Forwarded-For
Port: X-Forwarded-Port
Protocol: X-Forwarded-Proto
Network Load Balancer (NLB)
Forwards TCP & UDP (L4)
Millions of requests p/s
Lower latency than ALB
1 Static IP per AZ
Supports assigning Elastic IP (useful for whitelist)
Target Groups
EC2 instances
Private IP addresses
Application Load Balancer
Health Checks
TCP
HTTP
HTTPS
Gateway Load Balancer (GLB)
Deploy, scale & manage 3rd party network virtual appliances
Use cases
Firewalls
Intrusion Detection & Prevention Systems (IDS/IPS)
Deep Packet Inspection Systems
Payload manipulation
Forwards IP packets (L3)
Uses GENEVE protocol (port 6081)
Combines
Transparent Network Gateway
Load Balancer
Target groups
EC2 instances
Private IP addresses
Security groups
Allow port 80 & 443 to LB from anywhere
Allow port 80 / 443 to instance only from LB Security Group
Sticky Sessions
A.k.a Session Affinity
Same client sticks to same instance
Works for CLB, ALB, NLB
Used to ensure client doesn't lose session data
Can imbalance load over time
Cookie names
Application based
Custom cookie
Generate by target
Can include any attributes
Cookie name specified for each target group
Don't use
AWSALB
AWSALBAPP
AWSALBTG
Application cookie
Generate by LB
Named AWSALBAPP
Duration based
Generated by LB
Name is AWSALB for ABL
Name is AWSELB for CLB
Cross-Zone Load Balancing
ALB
Enabled by default
Inter AZ traffic is free
NLB / GLB
Disabled by default
Inter AZ traffic has costs
CLB
Disabled by default
Inter AZ traffic is free
SSL / TLS
TLS = Transport Layer Security
SSL = Secure Sockets Layer
CA = Certificate Authority
Issues public certificates
Load Balancer SSL
Uses X.509 certs
Manage with AWS Certificate Manager (ACM)
Can upload own certificates
HTTPS Listener
Must have default cert
Can add additional for other domains
Clients can use SNI (Server Name Indication)
Security policy controls SSL/TLS versions allowed
CLB
One SSL cert
No SNI support
ALB & NLB
Multiple listeners = Multiple SSL certs
Uses Server Name Indication (SNI)
Server returns correct certificate
Client indicates the hostname of the target on initial SSL handshake
One server with multiple SSL certs
Deregistration delay
Was named "Connection Draining" on CLB
Allows in-flight request to complete while target is de-registering
Stops sending requests to de-registering instance
1 - 3600s (default 300s)
Set to 0s to disable
Auto Scaling Group (ASG)
Goals
Scale out to match increased load
Scale in to match decreased load
Ensure min & max instances are not exceeded
Auto register new instances to load balancer
Auto recreate instance if it becomes unhealthy
No cost to using ASG
Attributes
Launch Template
AMI
Instance type
EC2 User Data
EBS Volumes
Security Groups
SSH Key pair
IAM Roles
Network information
Load Balancer info
Min Size
Max Size
Initial capacity
Scaling policies
CloudWatch Alarms
Can scale if alarm triggers
Alarm monitors metric
Metrics such as AVG CPU is for entire ASG
Dynamic Scaling Policies
Target tracking
Example: ASG average CPU should target 40%
Simple / Step scaling
CloudWatch alarm adds / removes specific number of instances
Schedule Actions
Schedule min / max capacity to increase / decrease on a schedule
Can base on predictive forecasts to get "Predictive scaling"
Metrics to scale on
CPUUtilization
RequestCountPerTarget
Average network in / out
Any custom metric
Scaling cooldowns
Default 300s
ASG won't launch / terminate during cooldown