AWS SAA-02
IAM
SQS(simple queue service)
Feature
Centralised control of your AWS account
Shared Access to your AWS account
Granular permission(可設定who access the resource who doesn’t access )
Identity Federation(including Activate Directory, Facebook…)
Multifactor Authentication
Provider temporary access for users/devices and service where necessary
Allow you to set up your own password rotation policy
Integrate with many different AWS service
Support PCI DSS Compliance (credict card)
Consist
User
Group
Role
Polices
JSON file
Universal
New Users No permission
Access Key ID & Secret Access Keys to login in to the console. Only can use to access AWS via the APIs and CLI.
AWS Directory Service
Connect AWS resources with Microsoft on-premise AD
Active Directory
On-premises directory service
Group policies
Standalone directory in the cloud
Three Microsoft Compatible service
Microsoft managed AD
Simple AD
AD Connector
directory gateway(proxy) for on-premises AD
Not AD Compatible
Cloud directory
Cognito user pool
Effect/Action/Resource
Amazon Resource Name(ARN)
arn:{aws/aws-cn}:{service}:{region}:{account_id}
example: arn:aws:s3::mybucket:
Effect either Allow or Deny ,Deny會取代Allow when 同時出現時
Resource Access Manager(RAM)
allows resource sharing between accounts
which AWS resources can I share
App Mesh
Aurora
CodeBuild
EC2
EC2 Image Builder
License Manager
Resource Group
Route 53
SAML
SSo
S3(Simple Storage service)
simple key-value store
Basic know how
單一檔案limit 0 bytes ~ 5TB
Object-based
Bucket DNS
Universal Namespace
Key
Static Website hosting
s3-website dash (-) Region ‐ http://bucket-name.s3-website-Region.amazonaws.com s3-website dot (.) Region ‐ http://bucket-name.s3-website.Region.amazonaws.com
Enhance security by "Access Control List" & "Bucket Policy"
Data Consistency Model
Read after consistency for PUTS of new Object
Eventually Consistency for overwrite PUTS and DELETES (can take some time to propagate)
object name
Value
基本上就是此檔案的資料本身
Version ID
作為版本控管之用
Metadata
額外用來記錄 object 相關資訊的資料
Storage Tiers/Classes
S3 Standard
99.99% availability,
99.999999999% durability
frequently accessed data
S3-IA
S3 One zone-IA
S3 - Intelligent Tiering
Glacier
S3 Glacier Deep Archive
99.9% availability,
99.999999999% durability
99.9% availability,
99.999999999% durability
long-lived, but less frequently accessed data
99.5% availability,
99.999999999% durability
long-lived, but less frequently accessed data
minimum storage period of 30 days
file backups larger than 128 KB
requires minimum storage period of 30 days
backups larger than 128 KB
move to standard or IA depend on access or not within 30day
requires minimum storage period of 90 days
long-term archive and digital preservation
limits the minimum size of the object to 40 KB
minimum storage period of 180 days
limits the minimum size of the object to 40 KB
RRS(Reduced Redundancy Storage)
99.99% availability,
99.99% durability
Charge for S3
Storage
Requests
Storage Management Pricing
Data Transfer Pricing
Cross Region
Encryption
S3 encryption In Transit
SSL/TLS
Encryption At Rest (Server side)
S3 Managed keys- SSE-S3
AWS key Management Service, Managed Keys-SSE-KMS
Server Side Encryption With Customer Provided Keys-SSE-C
Client Side Encryption
Master key(AES-256) 進行加密
全由 AWS 託管
S3 Object Lock
using a write once, read many (WORM)
Object lock come in two modes:
Governance mode
Compliance mode
users can't overwrite or delete
objects version can't be overwritten or deleted by any user,
S3 Glacier Vault Lock
specify controls such as WORM in vault Lock policy and lock the policy from future edits
S3 Performance
high number of requests: 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second prefix
multipart upload
when uploading files to S3
s3 byte-range fetches
when downloading files to S3
S3 Select
Retrieve only a subset of data by using simple SQL expression
Glacier Select
allows you to run SQL queries against Glacier directly
AWS Organization
Always enable multi-factor authentication on root account
Enable/Disable AWS services using Service Control Policies (SCP) either on OU or on individual accounts
Sharing S3 Buckets
3 Different ways to share S3 buckets across accounts
Using bucket Policies & IAM
Using bucket ACLs & IAM(individual object). Programmatic Access Only
Cross-account IAM Roles(Programmatic AND Console access)
S3 Transfer Acceleration
Using cloudfront edge network to 加速upload to S3
AWS DataSync
Move large and huge amounts on-premises to AWS
CloudFront
Only Global region
Key terminology
Edge Location
Origin
Distribution
This can be either an S3 bucket, an EC2 instance, an Elastic Load Balancer or Route53.
Two different types of distribution
Web Distribution
RTMP
Used for Media Streaming
used for Website
not just READ only –you can write to them
Signed URL's and Cookies
Use signed URLs/cookies when you want to secure content
signed URL
1 file = 1 URL
signed cookie
1 cookie = multiple files
AWS KMS provides an audit trail
Athena vs Macie
Athena
interactive query service which enables you to analyze and query data located in S3
Serverless
Macie
Machine Learning and NLP(Natural Language Processing) to discover, classify and protect sensitive data stored in S3
Storage Gateway
File Gateway(NFS)
Volume Gateway(iSCSI)
Tape Gateway(VTL)
Stored volumes
Cached volumes
used for backup
For flat files, stored directly on S3
Entire Dataset is stored on site and is asynchronously backed up to S3.
Entire Dataset is stored on S3 and the most frequently accessed data is cached on site.
1G – 16TB in size for stored volumes
Database
Relation Database(RDS)
Two feature
Multi-AZ- for Disaster Recovery, primary and standby by synchronize
Read Replica – For Performance, for scalability
Including SQL server , Oracle, MySQL server, PostgreSQL, Aurora, MariaDB ( SOPAMM)
OTLP (Online Transaction Processing)
OLAP(Online Analytics Processing)
Amazon’s Data Warehouse- Redshift
SQL server , Oracle, MySQL server, PostgreSQL, Aurora, MariaDB ( SOPAMM)
can’t access this RDS virtual machine
NOT Serverless. (with the exception of Aurora)
Two backup type
Automated Backups
“retention period” (1 and 35 days)
Enable by default
Database Snapshots
manual
They are stored even after you delete the original RDS instance
Restore Backups
will be a new RDS instance with a new DNS endpoint.
Multi-AZ
for Disaster Recovery only
SQL Server, Oracle ,MySQL Server ,PostgreSQL, MariaDB(SOPMM)
Read Replicas
Increase performance, for scalability
MariaDB, Microsoft SQL Server, MySQL, Oracle, and PostgreSQL(SOPMM)
Must have automatic backups turned on in order to deploy a read replica
have up to 5 read replica copies
Can be Multi-AZ
Can be in different regions
Encryption At Rest
Automated backup, read replicas, and snapshots are encrypted
supported for MySQL, Oracle, SQL Server, PostgreSQL ,MariaDB & Aurora (AOPMM)
using the AWS Key management Service(KMS)
DynamoDB
Basic
Stored on SSD storage
Spread across 3 geographically distinct data centres
Eventual Consistent Reads(Default) 1 秒
Strongly Consistent Reads less than 1秒
DynamoDB Accelerator(DAX)
Fully managed
Highly available
in-memory cache
Transaction
multiple “all-or-nothing ” operations
Financial transaction
Up to 25 items or 4 MB of data
On Demand Capacity
Pay-per-request pricing
No minimum capacity
No charge for read/write - only storage and backups
Backup and Restore
Can’t do backup or restore across region
consistent within seconds and retained until deleted
Point-in-Time recovery(PITR)
No enable by default
Protects against accidental write or delete
Restore to any point in the last 35 days
latest restorable: five minutes in the past
Stream
Stored for 24 hour
DMS(data migrate service)
DynamoDB is not support in source Database
Security
Encryption at rest using KMS
Redshift
data warehouse service
Massively Parallel Processing(MPP)
Configured
Single node(160 Gb)
Multi-Node
Leader Node
Compute Node
Backup
Enabled by default with a 1 day retention period
Maximum retention period is 35 days
Redshift always attempts to maintain at least three copies of your data (the original and replica on the compute nodes and a backup in Amazon S3)
asynchronously replicate your snapshots to S3 in another region for disaster recovery
Priced
Backup
Compute Node Hours (only charged CN hours)
Data transfer(only within a VPC, not outside it )
Security
Encrypted in transit using SSL
Encrypted at rest using AES-256 encryption
AWS KMS
Availability
Current only available in 1 AZ (can no have multi AZ )
Can restore snapshot to new AZs
Aurora
Basic
Start with 10G, Scale 10G ~ 64T(storage Autoscaling)
2 copies of your data is contained in each availability zone, with minimum of 3 availability zones. 6 copies of your data
Scaling
Replicas
Aurora Replicas (currently 15)
MySQL Read Replicas(currently 5)
PostgresQL(currently 1)
Automated failover is only available with Aurora Replicas
Backup
Automated backup
Do not impact database performance
You can also take snapshots with Aurora
not impact on performance
share Aurora Snapshots with other AWS account
Amazon Aurora Serverless
Elasticache
Memcached
Redis
increase database and web application performance
Can do backups and restore
If you need to scale horizontally, use Memcached
Database Migration Services (DMS)
Type of DMS Migrations
homogenous
heterogenous
oracle -> oracle
SQL Server -> Amazon Aurora
will need the AWS Schema Conversion Tool(SCT)
Caching Strategies on AWS
CloudFront
API Gateway
ElasticCache - Memcached and Redis
DynamoDB Accelerator(DAX)
EMR
industry-leading cloud big data for processing vast amounts of data using open-source tool such as Apache, Spark, Apache Hive,
Consists of a Master node, a core node and (optionally) a task node
click to edit
log data is stored on the master node
can configure replication to S3 on five-minute intervals for all log data from the master node ; however ,this can only be configured when creating the cluster for the first time.
Oracle , SQL server have limit to number per instance
NoSQL database
a much lower price point,
three times better than PostgreSQL
five time better performance than MySQL
are always enabled by default
be used to store message while waiting for a computer to process them
Two type of Queue
Standard Queues(default)
FIFO queues
能保证顺序
Visibility timeout
default visibility timeout for a message is 30 seconds (minimum is 0 seconds, maximum is 12 hour )
SQS is a way to de-couple you infrastructure
SQS is pull based, not push based
Messages are 256 kb in size
Message can be kept in the queue from 1 minute to 14 days, the default retention period is 4 days
not guaranteed and messages can be delivered more than once.
strictly maintained and messages are delivered only once
visibility timeout 設定太小，可能工作還沒處理結束，其他 consumer 又從 Queue 上面取下來作(等於傳送兩次)
若是 job 會執行超過 12 小時，可能 SQS 就不是適合拿來搭配的服務
Message timer
Delay queue
是用來設定 message 實際進入到 queue 中的延遲，超過設定時間後 message 才會真正進入到 queue 中
若是要故意讓進到 queue 中的 message 晚一點才能被 consumer 看見(initial invisibility)，可以透過設定 message timer 的方式來達成
Simple WorkFlow Service(SWF)
web service that makes it easy to coordinate work across distributed application components.
Workflow executions can last up to 1 year
Task-oriented API
Message-oriented API
Amazon SWF ensures that a task is assigned only once and is never duplicated
Amazon SQS, you need to handle duplicated messages and may also need to ensure that a message is processed only once
SWF Actors
Workflow Starters
An application that can initiate(start) a workflow
Deciders
Control the flow of activity tasks in a workflow execution
Activity Workers
Carry out the activity tasks
SNS( Simple Notification Service)
SNS benefit
push-based delivery(no polling)
Simple APIs and easy integration with applications
Flexible message delivery over multiple transport protocols
Elastic Transcoder
Media Transcoder in the cloud
Pay based on the minutes
API Gateway
Like front-door
101
Expose HTTPS endpoints to define a RESTful API
Serverless-ly connect to service like Lambda & DynamoDB
How do I deploy API Gateway
Send each API endpoint to a different target
Scale effortlesslyTrack and control usage by API key
Track and control usage by API key
Throttle requests to prevent attacks
You can log results to CloudWatch
Maintain multiple versions of your API(可測試Dev api)
Uses API Gateway domain, by default
Can use custom domain
Now supports AWS Certificate Manage: free SSL/TLS certs
API Gateway Caching
Enable API caching in Amazon API gateway to cache your endpoint’s response
Same Origin Policy
A web page可access 到B web page
Prevent Cross-Sit Scripting(XSS) attacks
Cross-origin resource sharing(CORS)
可限制從其他的domain 來的request
to increase performance
API Gateway is low cost and scales automatically.
CORS is enforced by client
Use multiple domains with API Gateway, ensure that you have enabled CORS on API Gateway.
Error - “Origin policy cannot be read at the remote resource?”
Kinesis
Amazon kinesis is a platform on AWS to send your streaming data to.
Kinesis makes it easy to load and analyze streaming data.
3 different types of Kinesis
Kinesis Streams
Kinesis Firehose
Kinesis Analytics
Shard 主要儲存kinesis streams data
Retention period: 24hour -> 7day
Has persistence default
不像 kinesis streaming 可有shard儲存
不care data persistence
提供開發者可以使用標準SQL查詢來分析即時串流資料,
可以發送至Redshift、Amazon S3、Amazon Elasticsearch Service或Kinesis Streams等其他AWS服務中使用
Web Identity Federation
lets you give your users access to AWS resources after they have successfully authenticated with a web-based identity provider like Amazon, Facebook, or Google
Cognito
User 會先跟 cogito 拿key 然後就可以直接存取
(FB的token換temporary key)
User pool
Enable provide temporary AWS credentials to access AWS services like S3 or DynamoDB
負責user account or password, 例如 email address, password…
拿JWT : JSON web token 去 User Pool
use “Push Synchroniztion ”push 更新使用著資訊到多台device
同時cognito也會利用SNS發送notification通知資料有被改變
SES (Simple mail service)
when a new message is added to the SQS queue, it will be hidden from consumer instance for a fixed period
Multi-AZ deployments utilize synchronous replication,
EC2
101
EC2 Pricing Model Option
On Demand
Reserved Pricing
Spot Pricing
Dedicated Host pricing
Allow you to pay a fixed rate by the hour(or by second) with no commitment
Is useful of
Users that want the low cost and flexibility of AWS EC2 without any up-front payment of long-term commitment
Application with short term, spiky , or unpredictable workload that cannot be interrupted.
Application being developed or tested on EC2 for the first time
require reserved capacity
Type
Standard Reserved instance
(up to 75% off on-demand)
Convertible Reserved instance
(up to 54% off on-demand)
Schedule Reserved instance
Is Useful of
Application with steady state or predictable usage
Application that require reserved capacity
User able to make upfront payments to reduce their total computing cost even further
Multifactor Authentication is required to delete
SQS will deliver your message at least once, but cannot guarantee that it will not create duplicates of that message. Additionally, SQS cannot guarantee message order.
Is Useful of
App that have flexibles start and end times
App that are only feasible at very low compute price
Users with urgent computing needs for large amount of additional capacity.
Noted
If the Spot instance is terminated by AWS EC2, you will not be charged for a partial hour of usage
If you terminate the instance yourself, you will be charged for any hour in which the instance ran.
簡單來講就是租用實體主機
Is Useful of
Regulatory requirements that may not support multi-tenant virtualization
Great for licensing which does not support multi-tenant or cloud deployments
Can be purchased On-Demand(hourly)
Can be purchased as a reservation for up to 70% off the On-Demand price
Instance types
FIGHT DR MC PX ZAU (fight this Dr Mark Pixy in Australia or in Austin.)
click to edit
I – for IOPS
G – Graphics
H – High Disk Throughput
T – Cheap general purpose
D – For Density
R – For RAM
M – Main choice for general purpose apps
C – for Compute
P – Graphics(think Pics)
X – Extreme Memory
Z – Extreme Memory AND CPU
A – Arm-based workloads
U – Bare Metal
F- for FPGA
On an EBS-backed instance, the default action is for the root EBS volume to be deleted when the instance is terminated
EBS(Amazon Elastic Block Store)
101
Provide persistent block storage volume for EC2 instance
Termination protection is turned off by default
ERS Root volumes of you DEFAULT AMI’s CAN be encrypted
you can also use a third party tool (such as bit locker etc) to encrypt the root volume, or this can be done when creating AMI’s in the AWS console or using the API
Additional volumes can be encrypted
EBS Storage 5 type
SSD
HDD
General Purpose(SSD)(gp2)
Provisioned IOPS(SSD) (io1)
General purpose, balance both price and performance
Throughput Optimized Hard Disk Drive(st1)
Cold Hard Disk Drive(sc1)
Magnetic (standard)
Use Case
Use Case
workloads where data is infrequently accessed
Volume Size
Use Case
File Server
Volume Size
500G - 1TB
Big Data & Data Warehouse, log processing
Volume Size
500G - 16TB
Use Case
Database
Volume Size
4GB - 16TB
Volume Size
1 GT - 16TB
always be in the same AZ as your EC2
Security group
All Inbound traffic is blocked by default
All Outbound traffic is allowed
Change to Security Groups take effect immediately
You can have any number of EC2 instance within a security group
You can have multiple security groups attached to EC2 instance
Security groups are STATEFUL,Do not have to configure outbound traffic
Cannot block specific IP address
Can specify only allowed rules, but not deny rules. (default deny everything)
Volume & Snapshot
101
刪除instance時會連同root volume一起刪掉,但是additional部分就不會
Snapshot exist on S3
Snapshot are incremental - mean that only the blocks that have changed since your last snapshot are moved to S3
If this is first snapshot, it may take some time to create
To create a snapshot for Amazon EBS volumes that server as root device, You should stop the instance before taking the snapshot.
Or Take a snap while the instance is running.
You can create AMI’s from both Volumes and snapshots.
You can change EBS volume sizes on the fly, including changing the size and storage type
To migrate EBS
To move an EC2 volume from one AZ to another, take a snapshot of it, create an AMI from the snapshot and then use the AMI to launch the EC2 instance in a new AZ.
To move an EC2 volume from one region to another, take a snapshot of it, create an AMI from the snapshot and then copy the AMI from one region to other, then launch EC2 instance
Security
Snapshots of encrypted volume are encrypted automatically.
Volumes stored from encrypted snapshots are encrypted automatically
You can share snapshots, but only if they are unencrypted (因為encrypted key tied to your AWS account)
RAID = Redundant Array of Independent Disks.
click to edit
RAID 0- Striped, No Redundancy, Good Performance.
RAID 1- Mirrored, Redundancy
RAID 5 – Good for reads, bad write , AWS doesn’t recommend ever putting RAID 5’S on EBS
RAID 10 –Striped & Mirrored, Good Redundancy ,Good Performance
EBS vs Instance Store
All AMIs are categorized as either backed by Amazon EBS or backed by instance store
For EBS Volume
The root device for an instance launched from the AMI is an Amazon EBS volume created from an Amazon EBS snapshot.
For Instance Store Volumes
他是最直覺的資料儲存方式，就像我們平常使用電腦，將資料儲存在電腦硬碟中一樣
Boot from instance store 無法選擇多項的 instance type (e.g t2.micro.. )
cannot be stopped. If the underlying host fails, you will lose your data
called Ephemeral storage
EBS backend instance can be stopped
By default, both ROOT volume will be deleted on termination.However, with EBS volumes, you can tell AWS to keep the root device volume.
AMI type
Select AMI based on
click to edit
Region(see regions and Availability zone)
Operating System
Architecture(32bit 64-bit)
Launch Permission
Storage for the Root Device(Root Device Volume)
Instance Store(ephemeral Storage)
EBS Backed Volumes
ENI vs ENA vs EFA
ENI
Elastic Network Interface
essentially a virtual network card
EN
Enhanced Networking
Use single root I/O vituralization (SR-IOV) to provide high-performance lower CPU utilization on supported instance type
EFA
Elastic Fabric Adapter
network device that you attach to
your EC2 instance to accelerate high performance computing HPC and machine learning applications.
1 GB - 1TB
click to edit
How to involved to encrypt the root device volumes
- Create a snapshot of the unencrypted root device volume. (cannot created encrypted snapshot!!!)
2.Copy of the snapshot and select the encrypt option
3.Create AMI from the encrypted Snapshot
4.Use that AMI to launch new encrypted instance
Not good for
persistent workload
critical Job
Database
Useful Task
Big data and analytic
Containerized workload
CI/CD Testing
Web service
Image and media rendering
High-performance computing
EC2 Hibernate
operation system does not need to reboot because the in-memory state(RAM) is preserved
is Useful for
Long-running processes
Service that take time to initilize
available for On-Demand instance and Reserved Instance
Instance RAM must be less than 150 GB
Instance families include(CMR) C3,C4,C5,M4,M5,R3,R4,R5
Available for Windows, Amazon Linux 2AMI, and Ubuntu
Instance can’t be hibernated for more than 60 days
Cloudwatch
Monitoring service to monitor your AWS resource, monitory performance.
Compute
EC2 instances
Autoscaling Group
Elastic Load Balance
Route53 Health Checks
Storage & Content Delivery
EBS Volume
Storage Gateways
Cloudfront
Host Level Metrics Consist of:
CPU
Network
Disk
Status Check
CloudTrail
monitors API calls in the AWS platform.
(default)Standard Monitoring = 5 Minutes
Detailed Monitoring = 1 Minute.
is all about performance
Is all about auditing
AWS Well Architected (SPRC)
Security
Reliability
Performance Efficiency
Cost Optimization
While the first 1000 invalidation paths per month are free, additional invalidation paths are $0.005 per request
first-byte latency of 3-5 hours when retrieving data from Glacier.
Roles are more secure than your access key and secret access key on individual EC2 instances
Roles are easier to manage
Roles can be assigned to an EC2 instance after it is created using both the console & command line
Role are universal - you can use them in any region
Using Metadata
Curl http://169.254.169.254/latest/user-data/ (get user data)
EFS & FSx
EFS
You only pay for the storage you use(no pre-provisioning required)
Data is stored across multiple AZ’s within a region.
Read After Writer consistency.
Support the NFS version 4 (NFSv4) protocol.
Amazon FSx for Windows
同樣也是全託管(fully managed)服務，資料的 HA 也會自動保證(同 region 跨 AZ，也可以選擇 single AZ)
Amazon FSx for Lustre
這是強化版的共享檔案系統，可同時被 Windows & Linux 使用，總之就是高效能 + 可擴展
Placement Groups
influence the placement of a group of interdependent instances to meet the needs of your workload.
支援 Microsoft Distributed File System (DFS)
low latency、high throughtput/IOPS 等特性，用在像是機器學習、HPC
可與 S3 進行整合，資料處理完就可以直接存放到 S3
不支援 Microsoft Distributed File System (DFS)
Three type
Cluster
Spread
partition
Grouping of instance within a single Availability zone
Used for very low latency or high network throughput or both
選擇 instance type 時，至少要選擇有 10Gb 以上網路的 instance type 才能享受到 placement group 的所帶來的優勢
Can span multiple Availability Zones and regions
only have 7 instances per Availability Zone Group
Can’t span multiple Availability Zones
Individual Critical EC2 instances
This strategy is typically used by large distributed and replicated workloads, such as Hadoop, Cassandra, and Kafka
can be used to deploy large distributed and replicated workloads, such as HDFS, HBase, and Cassandra, across distinct racks
has its own set of racks
AWS recommend homogenous instances within it
You Can’t merge placement groups
you can move an existing instance(must be in the stopped state) into a placement group
You can move or remove an instance using the AWS CLI or on AWS SDK, you can’t do it via console yet
HPC
Data Transfer
Compute & Network service
what are some ways we can get our data into AWS
Snowball, snowmobile (terabytes/petabytes worth of data)
AWS DataSync
store on S3, EFS, FSx for Windows, etc.
Direct Connect
click to edit
EC2 instance that are GPU and CPU optimized
EC2 fleets(Spot instances or Spot Fleets)
Placement groups (cluster placement group)
Enhanced networking single root I/O virtualization(SR-IOV)
Elastic Network Adapter(ENA) or Intel 82599 Virtual Function(VF) interface
Elastic Fabric Adapter(EFA)
Storage
Instance-attached storage
EBS: Scale up to 64,000 IOPS with Provisioned IOPS (PIOPS)
Instance store: Scale to million of IOPS ; low latency
Network storage
Amazon S3
Orchestration and automation
Amazon EFS
Amazon FSx for Linux
AWS Batch
AWS ParallelCluster
Choose Reserved Instances for continuous persistent load
Choose Spot instances for fault tolerant and Spiky loads
provides high availability
Long polling
helps reduce the cost of using Amazon SQS, 因為 worker 對 SQS 送出的 request 相對少了
Reduce false empty responses by querying all
Return messages as soon as they become available.
Short Polling
worker 會持續向 SQS 送出 request，即使 SQS 中沒有任何訊息，也會回應 empty
等待中當 Queue 中有訊息時，SQS 才會回應並帶上訊息(或者是等待到 timeout)
當訊息到達 SQS queue 時，就會馬上被送出
two mechanisms
Elastic Network Adapter(ENA)
intel 82599 Virtual Funcation(VF) Interface
Using Amazon s3 bucket will help to save any amount of data
Default Setting
Encrypted is not enable
no bucket policy exists
Configure in S3 console
Configure Server access logging
Configure Life cycle policy
server access logging provides detailed records for the requests that are made to a bucket
S3 Update – Strong Read-After-Write Consistency for put and delete
Restrict access to files in cloudfront caches
With CORS support
allow cross-region access to your Amazon s3 resource
version enable
To undelete an object. you must delete the delete marker
click to edit
Working with VPC Peering in Amazon EFS
(EC2 instance security group inbound)allow SSH access to the instance on port 22
2049 Port on NFS Target security Group
Support encryption data at rest, It can only be done during EFS creation
Performance Mode
General Purpose performance mode
Max I/O performance mode
for latency-sensitive use cases(low latency)
higher latencies for file metadata operations.
uch as big data analysis, media processing, and genomic analysis
web serving environments, content management systems, home directories, and general file serving
Can integration type
Lambda Function
HTTP
Mock
AWS Service
VPC Link
Throttle Setting
Steady-state rate
Burst request
requests per second
maximum bucked size
If the caller sends 10,000 requests in the first millisecond, API Gateway serves 5,000 of those requests and throttles the rest in the one-second period
Can override stage settings on an individual method within a stage
Different ways of controlling access to AWS API Gateway
Resource Policies
Standard AWS IAM roles and policy
CORS
Lambda authorizer
Amazon Cognito User pool
client-side ssl certificates
Usage plans
Automatically protects (default) your backend system from DDOS attack
Cache Setting on Console
Cache capacity
encrypt cache data
Flush entire cache
Default CloudWatch logging enables API request logging
Enable Access logging to how the caller accesed the API
Use Data Lifecycle Manager(DLM) to automate the creating, retention, and deletion of snapshot
Some instance types do not support instance store volume
Instance store data is lost
Underlying disk drive fails
instance stopped
Instance terminates
Encrypt EBS Volume are supported only instance type
Can enable encryption while copying a snapshot from an unencrypted snapshot
CloudWatch Metric
VolumeReadBytes
VolumeWriteOps
Volumethroughputpercentage
Support working with schema change
Asynchronous
Used infrequently, not throughout
Data stored in JSON format
Can Cross-Region Snapshot
Turn on a Trail across all regions:
CloudTrail will deliver log files from all regions to the Amazon S3 bucket
Create a Storage volume snapshot of the entire Database instance
Can Enable sharing with AWS Organizations
Can share the resource with another Organization in RAM
Can enable the verification during the final cut-over from on-premise to AWS
Pre-signed URL
The pre-signed URLs are useful if you want your user/customer to be able to upload a specific object to your bucket, but you don't require them to have AWS security credentials or permissions.
Case: When you want to give temporary access to users for S3 bucket
Origin Access Identity(OAI)
Restricting Access to Amazon S3 Content
easy to collect process, and analyze real-time, streaming data
Use Enhanced VPC Routing to force all COPY and UNLOAD through your VPC and no through the Internet
Snowball
Used for offline data transfer between on-premise & s3 bucket
Provide access only to authorized users for a specified time period
Can add an OIDC IdP to your user pool in the AWS Management Console
Object need to be stored in an S3 bucket with CSV, JSON or Apache Parquet format
GZIP & BZIPs compression is supported with CSV or JSON format with server-side encryption
launched with an EBS root volume
Cannot in an Auto Scaling group or used by Amazon ECS
Three types of Savings Plans (not support RDS)
Compute Savings Plans
help to reduce your costs by up to 66%.
provide the most flexibility
EC2 Instance Savings Plans
provide the lowest prices
offering savings up to 72%
Amazon SageMaker Savings Plans
help to reduce your costs by up to 64%
Enables your end-useds to use their existing corporate credential while accessing AWS applications
Microsoft Active directory compaible directory from AWS Directory Service
click to edit
user data can exec script and limited to 16 KB
master account cannot be removed from an AWS Organization
By default, CloudTrail event log files are encrypted using Amazon server-side encryption (SSE)
Expedited Retrievals
allow you to quickly access your data when occasional urgent requests are required for a subnet of archives
the data is available within 1-5 min
Standard retrievals
typically complete within 3 – 5 hours.
Security group ID can be configured as source
Support low latency and high throughput
no high-performance
Support low latency and high throughput and high-performance
delimiter character must always be a "&"
Can enable Auto Scaling, can automatically increase its write capacity
prevent objects from being deleted or overwritten for a fixed amount of time
Each object in an S3 bucket can have a user-defined storage class
requires an EC2 instance to be an encrypted EBS-backed instance.
可改變的東西比較多
Elastic IP address is for use in a specific region only
determines how instances are placed on underlying hardware
provide access to distinct racks
HSM to manage the top-level encryption keys
If an EBS volume is the root device of an instance, you must stop the instance before you can detach the volume
frequently accessed
following types of data are encrypted:
Data at rest inside the volume
All data moving between the volume and the instance
All snapshots created from the volume
click to edit
only available on certain instance types
Use signed URLs in the following cases
you want to restrict access to individual file
Your user are using a client that doesn't support cookies
Bulk retrievals
5 to 12 hours
Standard retrievals
12 hours
Bulk retrievals
48 hours
Important note for EC2 metrics:
CloudWatch does not collect memory utilization and disk space usage metrics right from the get go. You need to install CloudWatch Agent in your instances first to retrieve these metrics.
One EFA can be attached to an Amazon EC2 instance
Hibernate to start, Only Public IPv4 is allocated with new IP while Private IPv4 and any IPv6 are retained
Fit for long-running task
可從cloudwatch metric的ApproximateNumberOfMessagesVisible 知道q數量是否變多
如果bid price>spot price,費用就會拿spot price＊台數算,但後來如果bid price < spot price的話就會終止
use existing AD deployed at on-premise server
within a rack, if the rack fails, all instance fail at the same time
Max IOPS
500
Max IOPs
250
Max IOPS
64,000
Max IOPS
12,600
Step function is short-running task
如果超過 will automatically manage performance at this scale