Please enable JavaScript.

Coggle requires JavaScript to display documents.

AWS, AWS Regions - Coggle Diagram

- - - - you get Enhanced security with CloudFront OAI (Origin Access Identity)
        
        when Creating a cloudfront distribution bucket policy can be updated automatically to allow it to access S3
      - Cloudfront can be used as an ingress (to upload files to S3)
    - - ALB
      - EC2 Instance
      - S3 website (must enable static S3 website)
      - any HTTP backend
    - - **
        1.Client requests a file
        
        edge location forwards request to origin with included query strings and headers(e.g. S3) over private AWS network
        
        origin sends back requested items
        
        egde location caches results
        
        client makes similar request
        
        edge location checks cache first
        **
    - - EC2 instances must be public (publicly accessible from HTTP standpoint)
      - SG must allow public IP of edge locations (egde locations ip can be found on internet)
      - ALB must be public
        
        backend EC2 CAN BE PRIVATE
        
        SG has to allow Edge locations public ips traffic
  - - - manage by whitelists & blacklists
  - - - Global edge network
      - Files are tcaches for a TTL (maybe a day)
      - Great for static content that must be available everywhere
    - - Must be setup for each region you want replication to happen
      - files are updated in near real time
      - read only
      - Great for dynamic content that needs to be available at low-latency in few regions
  - - - Shared content (movie, music) - short ~few minutes
      - private content (private to the user) you can make it last for years
    - - URL - access to individual files (one signed URL per file)
      - cookie - access to multiplefiles (one signed cookie for many files)
  - - - Allow access to a path, no matter the origin
      - Account wide key-pair, only the root canam anage i t
      - can filter by ip, path, date, expiration
      - can leverage caching features
    - - Issue a request as the person who pre-signed the URL
      - Uses the IAM key of the signing IAM principal
      - Limited lifetime
      - S3 must be public
  - - - cost of data out per edge location varies
      - the more data transfered out per edge location the lower the cost
      - reduce edge locations for cost recution
      - three price classes
        
        All: all regions - best performance
        
        Price class 200: most regions, but excludes the most expensive regions
        
        NA, EU, Africa, South Asia
        
        Price class 100: only the least expensive regions
        
        NA & EU
    - - route to different kidn of origins based on content-type
        
        based on path patterns /images/ (S3 origin) api/ (e.g. ALB content)
      - Origin groups to increase HA and do failover
        
        Origin group: one primary, one secondary origin
        
        if primary fails, second is used
        
        can be used with S3 if enabled replication on bucket (CRR)
    - - protect user sensitive info through app stack
      - adds an additional layer of security along HTTPs
      - sensitive info encrypted at the edge close to user
      - asymetric encryption
      - Usage
        
        specify set of fields in POST requests that you want to be encrypted (up to 10 fields)
        
        client sends request with fields (e.g credit card info)
        edge location encrypt using pub key and send to CF via https
        CF send to ALB, ALB forward to web servers
        WS decrypt with private key
        
        specify the public key to encrypt them
  - - - Intelligent routing to lowest latency and fast regional failover
      - no issue with client cache (beacuse IP doesn't cahnge)
      - internal AWS entwork
      - Global Accelerator performs Health checks of apps
        
        helps make apps global (failover in less than 1min for unhealthy)
        
        good for DR
      - Security
        
        only 2 external IP need to eb whitelisted
        
        DDoS protection thank to AWS shield
    - - both use AWS global network and its edge locations around the world
      - both integrate with AWS shield for DDos protection
      - CF
        
        Improved performance for both cache-able content(e.g. images and videos)
        
        Dynamic content (e.g. API acceleration and dynamic site delivery)
        
        content served at the edge
      - GA
        
        improved performance for a wide range of apps over TCP or UDP
        
        proxying packets at the edge to apps running in one or more AWS regions
        
        Good for non-http, e.g. gaming (UDP), IoT (MQTT) or Voice over IP
        
        Good for HTTP use cases that require static IP address or fast regional failover
- - - - Hardware Key Fob MFA Device
      - Hardware Key Fob MFA Device for AWS GovCloud (US)
- - - - Application, colelctio of ERlastic Beanstalk components (environments, versions, configurations)
      - Application version
        
        iteration of app code
      - Environment
        
        Collection of AWS resources, running an app version (only one app version at a time)
        
        Tiers Web server environment tier & worker environment tier
        
        you can create many evns, (dev test prod )
- - - - Domain/subdomain name e.g. example.com
      - Record type - A, AAAA
        
        R53 supports following DNS record types A/AAAA/CNAME/NS :warning:
        
        A maps to IPv4
        
        AAAA maps to IPv6
        
        CNAME maps a hostname to another hostname
        
        target is a domain name which must have an A or AAAA record
        
        cant create a CNAME record for the top node of a DNS namespace (Zone Apex)
        
        Example, yo ucan't create for example.com but can for www.example.com
        
        NS Name Servers for Hosted zone
        
        control how traffic is routed for a domain
        
        Hosted Zone
        
        A container for records that define how to route traffic to a domain and its subdomains
        
        Public Hosted Zones contain records that specify how to route traffic on the Internet (public domain names)
        app.publicdomain.com
        
        Private Hosted Zones records specifying how you route traffic within one or more VPC (private domain names) app.company.internal
        
        $0.50 per month per hosted zone :warning:
      - Value - 12.34.56.78
      - Routing Policy
      - TTL time to live amount of time the record caches at DNS Resolvers
        
        high TTL e.g. 24h - less traffic on R53
        possibly outdated records
        
        low TTL more traffic on R53 - more cost :warning:
        outdated for less time
        
        except for Alias record, TTL is mandatory for each DNS record
      - Alias free of charge
        
        Alias vs CNAME :warning:
        
        CNAME points hostname to any other hostname
        ONLY FOR NON ROOT DOMAIN
        e.g. for domain name something.domain.com you own
        posible to do aka.something.domain.com
        
        Alias Points hostname to AWS resource
        work for root and non root domain aka.mydomain.com
        free :warning:
        
        always A or AAAA
        
        Unlike CNAME is can be used for the top node of a DNS namespace (Zone Apex) e.g. example.com
        
        alias record targets :warning:
        
        ELB
        
        CloudFront Distribution
        
        API Gateway
        
        Elastic Beanstalk environments
        
        S3 websites
        
        VPC Interface Endpoints
        
        Global Accelerator accelerator
        
        R53 record in same hosted zone
        
        YOU CANNOT SET AN ALIAS RECORD FOR AN EC2 DNS NAME
  - - - Typically route traffic to a single resource
      - can specify multiple values in same record
      - if multiple values are returned, a random one is chosen by the client
        
        every TTL timeout :warning:
      - when alias enabled, only one AWS reource can be specified
      - cant be associated with Health checks
    - - Control % of requests that go to each specific resource
      - assign each record a relative weight (they don't need to add up to 100)
      - DNS record must have the same name and type
      - can be associated with health checks
      - Use cases
        
        load balancing between regions
        
        testing new app versions be sending low amounts of traffic to them
      - assign weight of 0 to stop sending traffic to resource
      - if all records have weight 0, then all will be returned equally
    - - mandatory healthcheck, auto failover
    - - redirect to the resource that has the least latency close to us
      - super helpful when latency for users is a priority
      - latency is based on traffic between users and AWS Regsions
      - germany users may be directed to US, if thats the lowest latency
      - can be associated with health checks (has a failover capability)
    - - based on user location
      - specify location by continent, country or by US state, if thre's overlapping most precise loctation selected
      - should create a default record (in case no location match)
      - Use cases
        
        localization
        
        restrict content distribution
        
        load balancing
      - can be associated with health checks
    - - use when routing traffic to multiple resources
      - R53 return multiple values/resources
      - can be associated with health checks (return only values for healthy resources)
      - up to 8 healthy records are returned for each Multi-value query)
      - Multi value is not a substitute for having an ELB
    - - route traffic to resources based on geo location of users and resources
      - ability to shift more traffic to resources based on defined bias
        
        to change size of geo region specify bias values
        
        To expand (1-99) more traffic to resource
        
        to shrink -1 to -99 less traffic to resource
      - Resources can be
        
        AWS resources (specify AWS region
        
        non aws resources (specify latitude and longitude
      - must use R53 Traffic flow to use this feature :warning:
        
        Simplify the process of creating and maintaining records in large and complex configurations
        
        visual editor to manage complex routing decision trees
        
        configurations can eb saved as traffic flow policy
        
        can be applied to different R53 hosted zones (different domain names)
        
        supports versioning
      - kind of like weigted but with geolocation
    - - HTTP health checks only for public resources
      - health check => automated DNS failover
        
        can monitor an endpoint (app, server, other AWS resource)
        
        about 15 health checkers will check endpoint health
        
        healthy/unhealthy threshold (default 3)
        
        interval 30 sec ( can set to 10 sec - higher cost)
        
        supports HTTP, HTTPS, TCP
        
        if > 18% of health checksers report endpoint is healthy, R53 considers it healthy, otherwise unhealthy
        
        ability to choose which locations you want R53 to use
        
        health checks pass only when endpoint responds with 2xx or 3xx status
        
        health checks can be setup to pass / fail based on the text in first 5120 bytes of response
        
        must configure router/firewall to allow incoming requests from R53 health checkers
        
        can monitor other health checks (calculated health checks)
        
        combine many heal theck to a single health check
        
        you can use OR AND or NOT
        
        can monitor up to 256 child health checks
        
        specify how many of health checks need to pass to make the parent pass
        
        use case perform website maintenance without causing all health checks to fail
        
        can monitor CloudWatch Alarms (full control!) e.g. throttles of DynamoDB, alarms on RDS, cuystom metrics (helpful for private resources)
        
        R53 health checksers are outside of the VPC
        
        they can't access private dndpoints (private VPC or on premise resource)
      - integrated with CW metrics
  - - - example buy domain from goDaddy and use R53 to manage DNS records :warning:
        
        e.g. in go daddy speficy custom name servers
      - buy domain on 3rd party registrar, and still use R53 as DNS Service provider
        
        create hosted zone in R53
        
        Update NS records on 3rd party website to use R53 NS
- - - - Objects
        
        files have keys
        
        key is full path
        
        e.g s3://my-bucket/my_file.txt
        
        s3://my_bucket/my_folder1/another_folder/my_file.txt
        
        key is composed of prefix + object name
        
        max object size is 5tb
        
        cannot upload more than 5gb at a time must use multi-part upload for larger foles
        
        each object can have metadata KV pairs to add info t oobjects
        
        Tags useful for security/lifecycle
        
        version if versioning enabled at bucket level
        
        same key upload will increment version
        
        it is best to have versioning on
        
        protect agaionst unintended deleted
        
        easy rollback to prev version
        
        any not versioned file prior to enabling versioning will have version null when it is enabled
        
        suspending versioning does not delete previous versions
    - - no underscore
      - no uppercase
      - 3-63 chars long
      - not an I
      - must sart with lowercase letter or number
  - - - SSE-S3: encrypt using keys handled and managed by AWS
        
        Server side encryption
        
        AWES 256 enc
        
        must set header x-amz-server-side-encryption: AES256
      - SSE-KMS leverage AWS Key management Service to manage encryption keys
        
        must set header x-amz-server-side-encryption: aws:kms
        
        KMS Advantages: user control + audit trail
      - SSE-C: manage your own encryption keys
        
        encryption using data keys full managed by customer outside AWS
        
        Amazon S3 does not store enc key you provide
        
        HTTPS must be used
        
        encryption key must be provided in HTTP headers for every HTTP request made
      - Client Side Encryption
        
        encrypt file before upload
        
        clients must encrypt and decrypt data themselves
  - - - iam policies
        
        IAM principal can access an S3 object if
        
        user IAM permissions allow it OR resource policy allows it
        
        and no explicit DENY
      - resource based
        
        bucket policies
        
        what principles can and cannot do across buckets
        
        Object ACL
        
        Bucket ACL
        
        JSON based policies
        
        can be applied to buckets and objects
        
        Resources: buckets and objects
        
        Actions Set of API to allow or Deny
        
        Effect Allow/Deny
        
        Principal The account or user to apply policy to
        
        Use S3 Policy to
        
        grant public accessto bucket
        
        force objects to be encrypted at upload
        
        grant access to another accoutn (Cross account)
        
        Blocking public access
        
        blocking can be done by
        
        new access control lists (ACL)
        
        any access control lists (ACL)
        
        new public bucket ro access point policies
        
        block public and cross accoutn access to buckets and objects through any public bucket or access point policies
        
        these setting were created to prevent company data leaks
        
        if yo uknow your bucket should never be public, leave these on
        
        can be set at the account level
    - - supports VPN endpoints, for instances in VPC without www internet
    - - S3 access logs can be stored in other S3 bucket
      - API call can be logged in AWS cloudtrail
    - - MFA Delete, can be required in versioned buckets to delete objects
      - pre signed urls, valid only for limited time (ex premium video service for logged in users)
  - - - must be set up by ROOT account
      - you will need MFA to permanently delete object version
      - suspend versioning on bucket
      - not need MFA for enabling versioning, listing deleted versions
      - MFA delete can only be anbled using CLI at the moment
    - - one way to force encryption is to use a bucket policy and refuse API call to PUT S3 object without encryption headers
      - another way is to use "default encryption"
        
        bucket policies will be evaluated before "default encryption"
    - - for audit purpose, you may want to log all access to S3 buckets
      - any request mate to S3, from any account, authorized or not, will be logged into another S3 bucket
      - that data can be analyzed using data analysis tools or amazon Athena :warning:
      - :forbidden:do not set logging bucket to be the monitored bucket, it will create logging loop, and bucket will grow in size exponentially
    - - must enable versioning in source and dest
      - buckets can be in different accounts
      - Copying is asynchronous
      - must give proper IAM permissions to S3
      - CRR use cases compliance, lower latency access, replication accross accounts :warning: #
      - SRR use cases log aggregation, live replication between prod and test accounts :warning:
      - after enabling replication only new objects will get replicated
        
        but if you want to replicate old existing objects you can do so with S3 Batch Replication :warning:
        
        it will replicate existing objects, as well as those that failed replication
        
        for DELETE ops
        
        can repliucate delete markers from source to target (optional setting)
        
        deletions with a version Id are not replicated (to avoid malicious deletes)
        
        deletes will not be done on replica bucket
        
        there is not "chaining" of replication
        
        if bucket 1 has not replication into bucket 3, which has replication into bucket 3
        
        then objects created in bucket 1 are not replicated to bucket 3 :warning:
    - - Can generate pre signed URLs using SDK or CLI
        
        For downloads (easy, can use the CLI)
        
        For uploads (harder, must use the SDK)
      - Valid of a default of 3600 seconds (1hr) can change timeout with --expires in [TIME_BY_SECONDS] arg
      - users given a pre signed URL inherit permissions of the person who generated the URL for GET/PUT
      - examples
        
        allow only logged in users to download a premium video on your S3 bucket
        
        allow an ever changing list of users to download files by generating URLs dynamically
        
        Allow temporarily a user to uplaod a file to a precise location in a bucket
    - - Amazon S3 Standard - General Purpose
        
        99,99 Availability
        
        Used for frequent data access
        
        low latency and high throughput
        
        sustain 2 concurrent facility failures
        
        Use cases Big Data analytic, mobile & gaming apps, content distribution
      - Amazon S3 Standard-Infrequent Access (IA)
        
        99.9% Availability
        
        For disaster recovery, backups
        
        Lower cost than S3 Standard, For data less frequently accessed, but required rapid access when needed :warning:
      - Amazon S3 One Zone-Infrequent Access
        
        High durability 99.999999999% in single AZ, data lost when AZ is destroyed
        
        99.5% Availability
        
        Use cases storing secondary backup copies of non-premise data, or data you can recreate
        
        Lower cost than S3 Standard, For data less frequently accessed, but required rapid access when needed :warning:
      - Amazon S3 Glacier :warning:
        
        Amazon S3 Glacier Flexible Retrieval
        
        Minimum storage duration of 90 days
        
        Expedited (1-5 min) Standard (3-5h), bulk (5-12h) - free. :retrieval time
        
        Amazon S3 Glacier Deep Archive
        
        Minimum storage 180 days
        
        Standard (12h), bulk( 48h)
        
        Amazon S3 Glacier Instant Retrieval
        
        Milisecond retrieval, great for data accessed one a quarter
        
        Minimum storage duration of 90 days
        
        Low cost object storage meant for achiving/backup
        
        Pricing price for storage + object retrieval cost
      - Amazon S3 Intelligent Tiering
        
        :warning:small monthly monitoring and auto tiering fee
        
        moves objects automatically between Access Tiers based on usage
        
        No Retrieval charges :warning:
        
        Frequent access tier (automatic) default tier
        
        Infrequent Access tier (automatic) objects not accessed for 30 days
        
        Archive instant access tier (automatic) objects not accessed for 90 days
        
        Archive Access tier (optional) configurable from 90 days to 700+ days
        
        Deep archive Access tier (optional) config from 180 days to 700+ days
      - can move between classes manually or using S3 Lifecycle configurations
      - Durability & Availability :warning:
        
        Durability
        
        High durability 99.999999999% 11 9's of objects across multiple AZ
        
        if you store 10, 000, 000 objects with amazon S3 you can on average expect to lose a single object once every 10,000 years
        
        Same for all storage classes
        
        Availability
        
        Measures how readily available a service is
        
        varies depending on storage class
        
        Example S3 standard has 99,99% availability = not available 53 minutes/year
      - storage classes can be changed any time :warning:
        
        except for glacier, you have to copy to standard and then restore
    - - you can transition objects between storage classes
      - for infreq access, move them to STANDARD_IA
      - for archive objects, you don't need in real time, use GLACIER or DEEP_ARCHIVE
      - moving can be done manually or be automated with life-cycle configuration
      - transition actions rules when objects should be transitioned to another storage class
        
        e.g. move to Standard IA class 60 days after creation
        move to glacier for archiving after 6 months
      - expiration actions rules when objects are to be expired (deleted) after some time
        
        e.g. remove access log files after 1 year
        
        can be used to delete old file versions ( if versioning enabled)
        
        can be used to delete incomplete multi-part uploads #
      - Rules can be created for a certain prefix e.g. s3://mybucket/mp3/*
      - rules can be created for certain object tags (e.g. Department: Finance
    - - You can setup S3 Analytics to help determine when to transition objects from Stanrad to standard_IA
      - does not work for ONEZONE_IA or GLACIER
      - Report is updated daily
      - takes about 25h to 48h to first report
      - good first step to put together lifecycle rules (or improve them)
    - - automatic scales for high request rates, 100-200ms latency
      - app can achieve at least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per prefix ina bucket #
      - no limits to number of prefixes in a bucket
      - Example object path => prefix
        
        bucket/folder1/sub1/file => /folder1/sub1/
        
        bucket/folder1/sub2/file => /folder1/sub2/
        
        bucket/1/file => /1/
        
        bucket/2/file => /2/
        
        if you spread reads across all four prefixes evenly you can achieve 22,000 requests per second for GET and HEAD :warning:
      - KMS Limitation #
        
        if you use SSE KMS, you may be impacted by KMS limits
        
        when you Upload object it calls the GenerateDataKey KMS Api
        
        when you download it call the Decrypt KMS API
        
        Count towards the KMS quota per second (5500, 10000,30000 req based on region)
        
        if send more S3 requests you will be throttled :warning:
        
        you can request a quota increase using Service Quotas Console
      - Multi-part upload to increase performance #
        
        recommended for riles > 100MB
        must use for files >5GB
        
        can help parallelize uploads (speed up transfers)
      - S3 Transfer acceleration
        
        Increase transfer (up/down) speed by transferring file to an AWS edge location which will forward the data to the S3 bucket in the target region
        
        compatible with multi-part upload
      - S3 Byte-Range fetches
        
        Parallelize GETs by requesting specific byte ranges
        
        request files in parts (byte range fetches) in parallel possible
        
        can be used to only retrieve some partial data of a file, e.g. head of a file
        
        better resilience in case of failures
    - - Retrieve less data using SQL by performing server side filtering
      - Can filter by rows & columns (Simple SQL statements)
      - Less network transfer, less CPU cost client-side
      - Up to 400% faster, up to 80% cheaper :warning:
    - - S3:ObjectCreated
        S3:ObjectRemoved
        S3:ObjectRestore,
        S3:Replication
      - Object name filtering possible (*.jpg)
      - Use case generate thumbnails of images uploaded to S3
      - Use them to notify e.g. SNS, SQS, Lambda function
        
        typically events are delivered in seconds, but can take a minute or longer
        
        SQS requires you to set an access policy to allow incoming events
      - Can use Amazon EventBridge
        
        all events sent to AEB
        
        basing on rules it can forward events to Over 18 AWS services as destinations
    - - bucket owners will pay for S3 storage and data transfer costs associated with their bucket in general
      - instead you might want the requester to pay for networking cost for downloading data from the bucket :warning:
        
        helpful when sharing large datasets with other accounts
        
        requestor must be authenticated in AWS (cannot be anonymous) :warning:
    - - serverless query serivce to perform analytics against S3 objects
      - uses standard SQL language to query files
      - supports CSV, JSON ORC, Avro and Parquet (built on Presto)
      - Pricing :warning: $5 per TB of data scanned
        
        Use compressed or columnar data for cost-savings (less scanning)
      - Use cases Business Inteligence, analytics, reporting, analyze & query VPC Flow Logs, ELB logs, CloudTrail trails, etc
      - Exam tip serverless SQL S3 data analysis - Athena
    - - Adopt WORM (Write Once Read Many) model
      - Lock the policy for future edits (can no longer be changed)
      - Helpful for compliance and data retention
    - - WORM #
      - block object version deletion for a specified amount of time
      - Object retention
        
        Retention Period: fixed p[erios
        
        Legal Hold, some protection no expiry date
      - Modes
        
        Governance mode users cant overwrite or delete an object version or alter its lock settings unless they have special permissions
        
        Compliance mode: a protected object version can't be overwritten or deleted by any user, including root user in AWS account. when object is locked in compliance mode, its retention mode can't be changed. and its retention period cannot be shortened
- - - - control how traffic is allowed into or out of EC2
        Contain allow rules
        can reference by IP or SG
        
        INBOUND - from OTHER TO INSTANCE
        OUTBOUND - FROM INSTANCE TO OTHER
      - One SG can be attached to multiple instances
      - Region and VPC Locked
        will have to reacreate SGs in different regions/new VPC
      - TIMEOUT == SG ISSUE
        CONNECTION REFUSED - APP ISSUE
    - - General Purpose
        Compute Optimized
        Memory Optimized
        Accelerated Computing
        Storage Optimized
        Instance Features
        Measuring Instance Performance
        
        GP - good for diversity workloads e.g. web server or code repo
        Balanced compute memory and networking
        
        CO - high performance processors required e.g. game servers
        
        MO - Processing large data sets in mem
        
        SO - when often accessing local storage
        Relationald & NoSql DB
      - EC2Instances.info
    - - On-Demand Instanced, short workload, predictable pricing, pay by second
        
        Pay for what you use
        
        Linux & windows billing per second
        
        Other OS billing per hour
        
        highest cost no upfront payment
      - Reserved - 1 & 3 years
        
        Reserved Insances - long workloads
        
        Convertible Reserved Instanced - long workloads with flexible instances
        
        possibility to change instance type over time
        
        66% discount because more flexibility to what you can change
        EC2 Instance type, instance family, OS, scope tenancy
        
        Reserved instances are 72% discount compared to on-demand :question:
        reserve instance type region, tenancy, OS
        3 year discount bigger than for 1 year reservation
        All upfront payment biggest discount
        Reserved instance scope - regional or zonal (reserve capacity in an AZ)
        
        Use for steady-state uasage apps e.g. DB
        
        can be bought/sold on marketplace if not needed
      - Savings Plans - commitment to an amount of usage, long workload
        
        discount based on long term usage up to 72% like for reserved
        
        but you ahve to commit to type of usage e.g. $10/h for 1 or 3 years
        
        Usage over plan billed at on demand price
        
        Locked to instance family e.g M5,T2 and Region
        
        can change instance size m5.large, m5.2xlarge or OS
        
        can change tenancy
        Host, dedicated, default
      - Spot insances short workload, cheap , can loose instances so less reliable
        
        up to 90% discount compared to on-demand
        
        can lose instances if max price is less than current spot price
        
        most cost efficient
        
        workload resilient to failure - stopping will not negatively impact workload, able to recover
        
        not good for critical jobs and databases that are better if kept running at all times
        
        resume instance when spot price comes back to your declares maximum price
        
        2 minutes to stop/terminate instance after price goes above your decalred max price
        
        you can choose to SPOT BLOCK instance for 1-6h without interruptions, no interruptions - it is rare that it may be recalaimed by AWS
        
        not good for critical jobs and DB
      - Dedicated hosts book entire physical server, control instance placement
        visibility into lower level hardware
        
        allow to meet compliance requorements
        
        allows using server bound software licenses
        
        per socket per code per vm software licences
        
        On-demand - pay per second for each active dedicated host
        reserved for 1/3 years (no upfront, partial, all)
        It is most expensive option
        
        Per host billing
        targeted instance placement or automatic
        visibility of sockets, cored host id,
        add capacity using an allocation request
      - Dedicated Instances - no other customers will share hardware
        Own instance on own hardware
        
        hardware can be shared with instances in same account
        
        no control over instance placement (hardware can be moved after stop/start)
        
        automatic instance placement
        per instance billing
        use of dedicated physical servers
      - Capacity Reservations reserve On-demand instance capacity in AZ for any duration
        
        always have access to capacity
        
        no commitment, cancel anytime, no billing discounts
        
        To g et discounts combine with regional reserved and savings plans
        
        charged at on-demand rate all time whethere you run isntances o not
        
        for short term, uninterrupted workl;oads that need to be in a specific AZ
    - - 90% discount compared to On-demand
      - define max spot price while current price < max
      - spot block instances 1-6h
      - Spot Fleets :star:
        
        set of spot instances + optinal on -demand ionstances
        
        will try to meet target cpacity with price constrains
        
        define launch pool : instance type (m5.large), OS ,AZ
        
        can have multiple launch pools, and fleet will choose best one for the spot price to meet capacity
        
        Strategies to alloce Spot instances
        
        lowestPrice: will take lowest price from the pool (cost optimization, short workload)
        
        diversified: distributed across all pools (great availability, long workloads)
        
        capacityOptimized pool with optimal capacity for the number or instances
        
        allow extra saving, because automatically request spot instances with lowest price
  - - - managed NFS (network file system)
        
        can be mounted on many EC2 unlike EBS
        
        EFS work on EC2 isntances in multi-AZ
        
        highly available, scaleble and expensive
        
        3x gp2, pay per use
        
        Use cases
        content management, web serving , data sharing, workpredd
        
        uses NFSv4.1 protocol
        
        Uses SG to control access to EFS
        
        compatible with linux based AMI, not windows
        
        can use encryption at rest using KMS
        
        only POSIX file system (linux) that has standard API
        
        file system scales automatically, pay per use, no capacity planning
        
        Performance & storage classes :pen:
        
        EFS Scale
        1000s of concurrent NFS clients, 10GB+ throughput
        Grow to Petabyte scale network FS, autmatically
        
        Performance mode (set at creation)
        
        GP (default)
        
        latency use cases, web service, CMS etc
        
        max I/o higher latency, throughtput, highly parallel
        
        for big data, media processing
        
        Throughput mode
        
        bursting (1TB = 50MB/s + burst of up to 100MB/s
        
        Provisioned
        
        set throughtput regardless of sotrage size ex. 1GFB/s for 1TB storage
        
        Storage classes
        
        Storage tiers (lifecycle management feature) move file after N days
        
        Standard: for frequently accessed files
        
        Infrequent access (EFS-IA) :warning:
        
        cost to retrieve files, lower price to store
        
        enable EFS IA with a lifecycle policy
        
        EFS to multi AZ good for Prod, Regional
        
        one zone: One AZ, great for Dev, backup enabled by default, compatible with IA
        
        (EFS One Zone -IA) 90% discount in cost savings
    - - network drive, can be attached to instances while they run
        
        will use network so there may be latency
      - persist data when instance terminated to not lose data
      - can be mounted to one instance at a time :red_flag:
        
        BUT INSTANCE CAN HAVE MULTIPLE EBS ATTACHED
      - bound to specific AZ :warning:
        
        but EBS can be moved to another AZ with help of snapshot (backup) :red_flag:
        
        not required but recommended that volume is detached from istance :recycle:
        
        snapshots can be moved to EBS Snapshot Archive which is 75% cheaper :red_flag:
        
        takes 24-72h for restoring archive
        
        recycle bin for snapshots to be able to recover them after accidental deletion
        retention 1 day to 1 year
      - Analogy network USB stick
      - free tier :recycle: 30GB of free EBS of Genereral Purpose (SSD) or Magnetic per month
      - Provisioned capacity
        
        declare size in GB and IOPS - get billed for it
        
        can increase capacity or Iops
      - they do not have to be attached at all
      - delete on termination behaviour possible :red_flag:
        
        use case preserve root volume on instance termination
      - "limited" performance because it is a netwrok drive
      - Volume types :star:
        
        GP SSD
        
        gp2/gp3 (SSD) balances price and performance for wariety of workloads
        
        EC2 INSTANCES CAN ONLY USE GP2/3 and IO1/2 as root volume (where root os is running)
        
        cost effective, low latency
        
        1Gib-16Tib
        
        gp3
        
        3000 IOPS and throughput of 125MB/s
        
        can increase iops to 16000 and throughput to 1000mb/s insdependantly
        
        gp2
        
        small gp2 vol can burst iops to 3000
        
        size of volume and iops are linked, max iops 16000
        
        3 iops per GB - 5334 GB max IOPS
        
        Provisioned IOPS SSD
        
        io1/io2
        
        highest performance SSD vol for mission-critical low latency or high throughput workloads
        
        64000 iops (must be set to achieve max throughput)
        
        4GB-16TB
        
        for Critical business apps with sustained IOPS performance
        
        for apps that need more than 16000 iops
        
        io2 Block Express
        
        256k IOPS
        
        4GB-64TB
        
        sub milisecond latency
        
        4000MB/s throughput max
        
        all privisioned support amazon EBS multi-attach
        
        GREAT FOR db WORKLOADS SENSITIVE TO STORAGE PERFOMANCE
        
        EBS Multi-attach only for io1/2 family
        
        attach same EBS vol to multiple EC2 instances In same AZ
        
        each isntance will have full read & wrtie permissions to volume
        
        Use case
        high availability in clustered linux apps
        apps must manage concurrent write Operations :warning:
        
        must use a file systems thats cluster aware
        
        ST1 (HDD)
        
        low cost HDD vol designed for frequent access & high throughput instansive workloads
        
        CANNOT BE BOOT VOL
        
        BIG DATA, DATA WAREHOUSES, LOG PROCESSING
        
        MAX THROUGHPUT 500MB/S MAX IOPS 500
        
        SC1 (HDD)
        
        Lowest cost HDD vol for less frequent access workloads
        
        CANNOT BE BOOT VOL
        
        INFREQUENT ACCESS
        
        WHERE LOWEST COST IMPORTTANT
        
        MAX THROUGHPUT 250MB/S MAX IOPS 250
        
        EBS VOLUMES CHARACTERIZED IN SIZE | THROUGHPUT | IOPS :warning:
        
        vOLUME TYPES DOCS :check:
      - EBS encryption : :red_flag:
        
        data at rest ins encrypted inside volume
        
        data in flight moving between instance and volume is encrypted
        
        all snapshots encrypted
        
        volumes created fro msnapshot are encrypted
        
        enc/dec handled transparently without havign to do anything
        
        encryption has minimal impact on latency
        
        uses keys from KMS (AES-256)
        
        copying unencrypted snapshot allows encryption
        
        snapshots of encrypted volumes are encrypted
        
        HOW TO ENCRYPT UNENCRYPTED EBS VOL :warning:
        
        Create EBS Snapshot of volume
        
        Encrypt snapshot using copy
        
        Create new EBS vol from snapshot (volume will also be encrypted)
        
        attach encrypted volume to original instance
    - - High performance hardware disk
      - disk physically attached to server
      - better I/o performance
      - Ephemeral storage
        
        instance/instance store termination causes storage to be lost
      - good for buffer/ cache/ scratch data/ temporary content storage
      - risk of data loss if hardware fails
      - your responsibility to backup to not lose data
  - - - IPV regular ip 1.111.11.111
        
        3,7 billion addresses in public space
      - ipv6 1900:4545:3:200:f8ff:fe21:67cf
        
        for IOT
    - - Only 5 elastic ip in account available, can ask AWS to incerease
        
        should use random public ip and register DNS name to it
    - - cluster placement groupfor big data job to complete fast
        
        clustered instanced into a low latency group in single AZ
        
        great network
        
        if rack fails, all insances fail at the same time
        
        app that needs extremely low latencyt and high network speed
      - spread placement group
        
        spreads instances accross underlying hardware (max 7 isntances per group per AZ) critical applications
        
        minimize failure rist
        
        all instances on differnt hardware
        
        reduced risk of simulatneous failure
        
        ec2 on different hardware
        
        7 instances per AZ per placement group
        
        for APPS that need high availability
        Critical apps where instance failure must be isloated from each other
      - partition
        
        spreads insances across many different partitions that rely on different sets of racks within AZ scales to 100s of EC2 per group
        
        7 partitions per az in same region, each partition is a separated rack
        
        Up to 100s of EC2 instances
        
        instances do noth share racks with instances in other partitions
        
        partition failure can affect many EC2 but not other partitions
        
        ec2 get accesses to partition info as metadata
        
        good for kafka, cassandra, HDFS - partition aware big data apps
    - - logical component in VPC that represents a virtual network card
      - can have primaty private ipv4 and one or more seconrdary ipv4
      - one elastic ip per private ipv4
      - one public ipv4
      - one or more SG
      - can have a mac address
      - can have ENI independendly and attach them on the flue (move them) on EC2 instances for failover
      - bound to a AZ
      - READ THIS FOR MORE INFO ON ENI :!:
    - - RAM state preserved, boot much faster OS not stopper/restarted
        
        its like pausing
        
        Long running processing
        
        saving the RAM state
        
        Services that take time to init
        
        insance RAM size must be less than 150GB
        
        not available for bare metal isntances
        
        root volume must be EBS encrypted, not instance store and large
        
        available for on demand, reserved
        
        not longer than 60 days
      - RAM Stored on EBS (ebs must be large enough)
    - - next gen of ec2 isntances
        new virtualiuzation tech
        better performance
        better networking
        higher EBS (nitro for 64k IOPS max 32k non nitro
        better security
  - - - your own AMI
        
        AWS Marketplace AMI that someone else has made, they may be selling it
- - - - Data migration
        
        SnowCone
        
        small device
        
        small, portable computing,anywhere, rugged & secure, withstands harsh environments
        
        light 2,1kg
        
        device used for edge computing, storage, and data transfer
        
        8TB of usable storage
        
        Use snowcone where Snowball does not fit (space-constrained environment)
        
        must provide own battery and cables
        
        Can be sent back to AWS offline, or connect it to internet and use AWS DataSync to send data
        
        Snowball Edge (for data transfers)
        
        Physical data transport solution: move TBs or PBs of data in or out of AWS
        
        Alternative to moving data over the network and paying network fees
        
        Pay per data transfer job
        
        provide block storage and Amazon S3-compatible object storage
        
        Snowball Edge Storage Optimized
        
        80TB of HDD capacity for block volume and S3 compatible object storage
        
        Snowball Edge compute optimized
        
        42TB of HDD capacity for block volume and S3 compatible object storage
        
        Use cases: large data cloud migrations, DC decommission, disaster recovery
        
        large device
        
        Snowmobile
        
        truck :truck:
        
        each snowmobile tak 100PB of capacity
        
        better than snowball if you transfer more than 10PB
        
        offlice devices to perform data migrations from AWS
        
        if it takes more than a week to transfer data, these should be used
        
        Amazon sends device, instead of direct client upload, put data on device and send it back to Amazon, they will physicaly add data to destination (e.g. S3)
      - Edge computing
        
        Snowcone
        
        Snowball edge #
        
        process data while its being created in an edge location, truck on the road, ship on sea, mining station underground - places with limited or without internet access
        
        Use cases
        
        preprocess data
        
        machine learning at the edge
        
        if needed ship back the device to AWS (for transfering data for example)
    - - 2CPUs, 4GB RAM, wired or wireless access
      - USB-C power using a cord or the optional battery
    - - 52 vCPUs, 208 GiB of RAM
      - optional GPU (useful for video processing or machine learning)
      - 42TB usable storage
      - Snowball cannot import data directly to Glacier - instead use S3 in combination with life-cycle policy
    - - Up to 40 vCPUs, 80GB RAM
      - Object storage clustering available
    - - Long-term deployment options: 1 and 3 years discounted pricing
    - - historically snow devices needed you to use CLI
      - install on computer to manage your snow family device
        
        unlocking and configuring single or clustered devices, transferring files, launching and managing instances running on snow family devices
        
        monitor device metrics (storage capacity, active isntances on your device)
        
        Launch compatible AWS services on your devices (EC2, AWS DataSync, Network File System)
  - - - parallel distributed FS , for large scale computing
      - Lustre = Linux + cluster
      - machine learning, High Performance Computing (HPC)
      - Video processing, Financial Modeling, Electronic Design Automation
      - Scales up to 100s of GB/s, millions of IOPS, sub-ms latencies
      - Storage options
        
        SSD low latency, IOPS intensive workloads, small & random file operations
        
        HDD throughput intensive workloads, large & sequential file operations
      - Seamless integration with S3 :warning:
        
        can "read S3" as a file ssytem (through FSx)
        
        can write the output of computations back to S3 (through FSx)
      - Can be used from onpremise servers (VPN or Direct Connect)
    - - it is like EFS for POSIX system for linux systems
      - fully managed Windows file system share drive
      - SMB & Windows NTFS
      - Microsoft Active Directory integration, ACLS, user quotas
      - Can be mounted on Linux EC2 instances
      - Scale up to 10s of GB/s, millions of IOPS, 100s PB of data
      - Storage options
        
        SSD - latency sensitive (DB, media processing, data analytics)
        
        HDD - broad spectrum of workloads (home directory, CMS) where latency is less important
        
        Can be accessed from on-premise infrastructure (VPN or DirectConnect)
        
        HA can be configured to be multi-AZ
        
        you can backup data daily to S3
    - - Scratch File system
        
        Temp storage
        
        Data not replicated, doesnt persist when server fails
        
        high burst (6x faster, 200MBps per TiB
        
        Use Case: short term processing, optimize costs
      - Persistent file system
        
        Long-term storage
        
        Data replicated within same AZ
        
        Replace failed files within minutes if server failure
        
        Usage: long term processing, sensitive data
    - - local cache for frequently accessed data
      - native access to Amazon FSx for Windows File server
      - Windows native compatibility (SMB, NTFS, Active Directory)
      - Useful for group file sshares and home directories
  - - - this can be due to
        
        long cloud migrations
        
        Security requirements
        
        Compliance requirements
        
        IT strategy
      - can be used to expose S3 Data on-premises
    - - file Gateway
        
        configured S3 buckets are accessible using the NFS and SMB protocol
        
        supports S3 standard, S3 IA, S3 one zone IA
        
        bucket access using IAM roles for each File Gateway
        
        most recently used data is cached in the file gateway
        
        can be mounted on many servers
        
        integrated with Active Directory (AD) for user authentication
      - Volume gateway
        
        Block storage using iSCI protocol backed by S3
        
        backed by EBS snapshots which can help restore on-premises volumes
        
        Cached volumes low latency access to most recent data
        
        Stored volumes entire dataset on premise, scheduled backups to S3
      - Tape gateway
        
        Some companies have backup; processes using physical tapes
        
        with tape gateway, companies use the same processes but in the cloud
        
        Virtual Tape library (VTL) backed by amazon S3 and Glacier
        
        backup data using existing tape-based processes (and iSCSI interface)
        
        works with leading backup software vendors
    - - using storage gateway means you need on-prem virtualization
      - Otherwise, you can use a Storage gateway Hardware appliance
      - you can buy it on amazon.com
        
        install in on-premise infrastructure, and works with file, volume and tape gateway
        
        has the required CPU, memory, network, SSD cache resources
      - helpful for daily NFS backups in small data centers
    - - on-premises data to the cloud => storage gateway
      - file access/NFs - user auth with Active Directory - file gateway (backed by S3)
      - Volumes/ block storage / iSCSi - volume gateway ( backed by S3 with EBS snapshots
      - VTL tape solution / backup with iSCSI - Tape gateway (backed by S3 and Glacier
      - No on-premises virtualization - Hardware appliance
  - - - FTP, FTPS, SFTP
- - - - old generation /v1 /deprecated
        TCP (Layer 4) HTTP & HTTPS (Layer 7)
      - health check are tcp or http
      - fixed hostname xxx.region.elb.amazon.aws.com
      - Instances tab under instance status must be InService to know it works :explode:
        
        else unhealhy
        
        OutOfService wrong SG configuration and instance is not configured to take traffic from LB :fire:
    - - v2 HTTP HTTPS WebSocket
      - Layer 7 only (HTTP)
      - LB to multiple HTTP apps across machines (target groups)
      - LB to multiple apps on same machine (e.g. in containers)
      - can do HTTP to HTTPS redirects at LB level
      - routing based on path in url (like in nginx :star:) e.g. /users
        routing on hostname e.g. one.example.com & two.example.com
        routing based on query string, headers e.g. example.com/users?id=123
      - good for micro services and container based apps e.g. docker & amazon ECS
        
        has a port mapping feature to redirect to dynamic port in ECS
      - we would need multiple CLB per app to do what ALB can
      - target groups
        
        EC2 instances (that can be managed by ASG) - http
        
        ECS tasks (managed by ECS itself) -http
        
        Lambda functions HTTP request transalted into a JSON event
        
        IP address (only private)
        
        ALB can route to multiple target groups
        
        health checks at target group levelR
        
        other ALB as target group
      - has fixed hostname xxx.region.elb.amazonaws.com
      - app servers dont see ip of client directly
        
        true IP of client inserted in X-Forwarded-For header
        
        also port X-Forwarded-Port & proto X-Forwarded-Proto
    - - v2 TCP, TLS (Secure TCP, UDP
      - Layer 4
      - TCP & UDP traffic forwardin
        
        for extreme performance
      - millions of requests per second
      - less latency ~100ms vs 400ms for ALB
      - has one static IP per AZ supports assigning elastic IP (helpful for whitelisting specific IP
      - target groups
        
        ec2 isntances
        
        in EC2 SG we must specify to allow TCP/UDP/etc traffic into those instances
        
        HTTP TCP 80 0.0.0.0/0
        
        IP addresses (must be private)
        
        e.g. if you have a server in your own data center and want to route traffic to there
        
        Application LB
        
        e.g. for using its http routing features
    - - Operates at later 3 (network layer) ip protocol
      - deploy, scale and manage a fleet of 3rd party network virtual appliances in AWS
        
        e.g. firewalls, intrusion detection, prevention systems, deep packet inspection systems, payload manipulation
        
        Users > VPC route table routes traffic to GLB > sends traffic to target group of machines (e.g. EC2) for analysis (firewall, intrustion detection etc. > if they are happy with it they return or drop traffic to GLB that forwards it to the destination APP :explode: :explode:
      - Transparent Network Gateway single entry/exit for all traffic
      - Uses GENEVE protocol on port 6081 :warning: it means GLB in exam
      - target groups
        
        EC2
        
        IP addresses (private)
    - - HA across zones
      - separate public traffic from private
      - Enforce stickiness with cookies
      - costs less to setup own LB but will require more effort on your end
    - - making it possible to implement stickiness so same client always is redirected to same instance behind LB
        
        possible for CLB & ALB
        
        "cookie" used for stickiness has an expiration date you control
        
        USE CASE: make sure used does not lose session data
        
        enabling stickiness may bring imbalance to load over the backend EC2
      - tw otypes of cookie names
        
        app based cookie
        
        custom cookie generated by target
        can include any custom attributes required by app
        cookie name must be specified individually for each target group
        
        cannot use AWSALB AWSALBAPP OR AWSALBTG names for cookie (reserved for use by ELB
        
        Application cookie
        
        generated by LB
        
        cookie name is AWSALBAPP
        
        duration based cookies
        
        cookie generated by LB
        
        name is AWSALB for ALB AWSELB for CLB
      - in target group > edit attributes > enable stickiness :star:
    - - option each LB distributes load evenly across all instances in AZ
        
        without cross zone- each LB node distributes requests for instances in specific AZ e.g. node 1 only to AZ1 node 2 only do AZ2 :explode:- some instances in specific AZ will receive more traffic if they have more instances
      - ALB CZ always on, no charges for inter AZ data
      - NLB disabled by default
        
        charged for inter AZ data if enabled
      - CLB disabled by default
        
        no charges for inter AZ if enabled
    - - allow trafic between clients and LB to be encrypted in transit (in-flight encryption)
      - SSL secure socket layer used to encrypt connections
      - TLS Transport Layer security - newer version
      - SNI server name identication
        
        solved problem of loading multiple SSL certs onto one web server to serve multiple websites
        
        it is a newer protocol, that requires client to indicate the hostname of target server in the initial SSL handshake
        
        server will find correct cert, or return default one
        
        ONLY WORKS FOR ALB & NLB, CLOUDFRONT :warning:
        
        not for CLB beause old gen
    - - connection draining for CLB
      - deregistration delay for ALB & NLB
      - time to complete "in-flight" requests while instance is deregistering or unhealthy
        
        it will give instance some time before it stops sending requiests to EC2
      - 1-3600 seconds to complete requests default 300
        
        can disable so no draining is happenning (set to 0)
        
        set low value if requests are short (less than 1 second)
  - - - dynamic scaling policies
        
        Target tracking scaling
        
        most simple
        
        e.g I want average ASG cpu to be around 40% usage
        
        Simple/ step scaling
        
        e.g. when cloudwatch alam is triggered e.g. cpu >70, then add or remove instances
        
        Scheduled Actions
        
        scaling based on known usage patterns e.g. inscrease min capacity on fridays
      - predicive scaling
        
        forecast load and schedule ahead
      - good metrics to scale on
        
        CPU Utilization
        
        RequestPerCountPerTarget
        
        to make sure no of requests per EC2 is stable
        
        Average network IN/out if app is network bound
        
        any custom metric that you push using CloudWatch
    - - After scaling activity there is a cooldown period (default 300s)
        
        during that time ASG will not launch or terminate instances to allow metric stabilization
      - TIP use ready to use AMI to reduce configuration time to serve requests faster and reduce cooldown period
    - - find AZ with most number of instances
        
        if there are multiple to choose from delete one with oldest launch config
        
        will try to balance no of instances across AZ by default
      - Lifecycle hooksame for teminating
        
        eg install extra software, perform extra checks before instance is inservice
    - - both have AMI ID, instance type, key pair SG, other things used to launch EC2 ,tags userdata
      - Launch config is legacy
        
        must be recareted every time parameter changes are to be done
      - Launch Template
        
        recommended
        
        multiple versions
        
        partial reusable template
        
        can use T2 burst
- - - - in case server hijack data is safe
      - stored in a n encrypted form thanks to a key (usually a data key)
      - enc/dec keys must be managed somewhere and the server must have access to it
  - - - key replicated to other regions with same keyId
      - enc in one region dec in other
      - key rotation replicates to other regions
      - no need to re-encrypt when cross region
      - KMS mR are not global (primary + replicas)
      - each mr key is managed independently
      - use cases: global slient side enc, enc on global dynamo db, global aurora
        
        DynamoDB
        
        enc specific attributes client-side in dynamo table using Amazon dynamoDB enc client
        
        stops admins from decrypting
        
        when using global table data is replicated to another region
        
        client in another region is able to decrypt using Mr key
    - - when S3 rep is enabled all unenc and enc with SSE-S3 will be replicated by default
      - Objects enc with SSE-C (Customer provided key) never replicated
      - for object enc with SSE-KMs you need to enable the option and specify which KMS key to enc objects within target bucket
      - adapt the KMS key poliocy for target key
      - an IAM role with kms:Decrypt for the source KMs key and kms:Encrypt for target KMS key
        
        you might get KMs throttling errors, in which case you can ask for a Service Quotas increase
    - - SYMMETRIC AES-256 keys
        
        single encyption key that is used to enc and dec
        
        aws services that integrate with KMS use symetric CMKs (Customer managed Keys 1$/month)
        
        when we create or use sym key, yo unever get access to unencrypted key - it is held in KMS and api has to be used to work with it
        
        AWS managed key is free, CMK 1$/month
        
        CMK imported (must be 256 sym key) 1$/month
      - Asymetric (RSAA & ECC key pairs)
        
        pub (enc) and private (dec) key pair
        
        used for enc/dec or sign/verify operations
        
        pub key is downloadable, private cannot be accessed unencrypted
        
        use case: enc outside of AWS by users who cannot access KMS API
      - pay for API call to KMS $0.03/10000 calls
      - Automatic key rotation
        
        AWS-managed KMS key automatic every 1 year
        
        CMK (must be enabled) auto every 1 year
        
        Imported KMs key - manual rotation only possible using alias
      - KMS keys scoped per region
        
        Copying snapshots across region will cause Amazon to re-encrypt with a new Key in dest region
    - - controll access to KMs keys, similar to s3 bucket policied
      - difference: ytou cannot controll access w ithout them
      - default KMS key policy
        
        created if you don't provide specific KMs key policy
        
        complete access to the key to thew root use = entire AWS account
      - Custom KMs key policy
        
        define users, roles that can access the KMS key
        
        define who can administer the key
        
        useful for cross-acc access of ytou kms key
- - - - Postgres
        MySQL
        MariaDB
        Oracle
        MS SQL Server
        Aurota (AWS Proprietary db)
    - - OS Patching
        Automated Provisioning
      - continuous backups and restore to specific timestamp (point in time restore)
      - monitoring dashboards
      - read replicas for improved read performance
      - multi AZ setup for Disaster Recovery
      - Maintenance windows for upgrades
      - scaling horizontal and vertical
      - storage backed by EBS (gp2 or io1)
      - Cant SSH into instances
    - - automatically enabled
      - automated backups
      - daily full backup during maintenance window
      - t4ransaction logs backed up every 5mins
      - ability to restore to any point in time (from oldest backup to 5minago)
      - 7 days retention, can be increased to 35
      - DB Snapshots
        
        manually triggered by user
        
        retention of backup for as long as you want
    - - dynamic storage increase on RDS DB instance
      - when RDS detects you are running out of free db storage it scales automatically
      - avoid manually scalign db storage
      - you have to set MAX Storage thershold (max limit for DB storage)
      - auto modify storage if
        
        free storage less than 10%
        
        low storage lasts at least 5 mins
        
        6hrs have passed since last modif
      - useful for apps with unpredictable workload
    - - cost when data goes fro m one AZ to another
      - RDS read replicas in same region you dont pay fee
    - - SYNC replication
      - one DNS, auto app failover to stanby replica
      - increase availability
      - failover in case of loss of AZ network instance or storage failure
      - READ replicas can be setup as Multi AZ do Disaster Recovery
    - - possible to encrypt the master & read replicas with AWES KMS AES-256 encryption
      - encryption must be defined at launch time
      - If master not encrypted read replicas canno be encrypted
      - TDE transparent data encryption avaialble for Oracle and SQL Server**
      - in-flight encryption
        
        SSL certs
        
        provide SSL options with trust cert when connecting to DB
        
        To enforce SSL:
        
        PostgreSQL rds.force_ssl=1 in AWS RDS Console (Parameter Groups)
        
        MYSQL :check:
        
        GRANT USAGE ON . TO 'mysqluser'@'%' REQUIRE SSL;
      - Encryption Operations
        
        Encrypting RDS backups
        
        Snapshots of unencrypted RDS db are unencrypted
        
        snapshots of encrypted RDS dbs are encrypted
        
        make an encrypted snapshot - copy a snapshot into an encrypted one
        
        Encrypt unencrypted RDS db :warning:
        
        Create snapshot of unenc db
        
        Copy snapshot and enable encryption for the snapshot
        
        restore the db from encrypted snapshot
        
        migrate apps to new DB and delete old db
    - - RDS db usually deployed in private subnet, not public - make sure db not exposed to www
      - RDS security works via the help of SG, same as for EC2 - it controls which IP/SGs can communicate with RDS
      - Access management
        
        IAM policies help control who can manage AWS RDS via RDS API
        
        Username & password can be used to login
        
        IAM base auth can be used to login into RDS mMysql & PostgreSQL
        
        Only pgres & mysql
        
        no password jsut auth token obtained through IAM & RDS Api calls
        
        token lifetime 15mins
        
        benefits :check:
        
        in/out network must be encrypted using ssl
        
        iam to centrally manage users instead of DB
        
        levarage IAM roles and EC2 instance profiles for easy integration
  - - - your drivers will work as if Aurora was a postgres or MySql DB :warning:
    - - High Availability native :warning:
    - - 6 copies of data across 3 AZ
        
        4 copies out of 6 needed for writes
        3 copies out of 6 needed for reads
        so if 1 Az down we're still able to read & write
        
        self healing with p2p replication
        
        storeage is striped across 100s of volumes
      - One Aurora instance takes writes (it is master)
        
        if master down failover in less than 30s
        
        multi master possible, good when continuous writer availability is required
      - master + 15 read replicas to serve reads
        any of those can become master for failover :warning:
      - Cross region replication :warning:
    - - Writer endpoint (DNS endpoint directing to current maser instance)
      - auto scaling for read replicas is possible
      - Reader endpoint - connection load balancing to all read replicas :warning:
      - shared storage volume that auto expands for all write & read replicas of Aurora :warning:
    - - Auto failover
      - backup & recovery
      - isolation and security
      - industry compliance
      - push button scaling
      - auto patching with zero downtime
      - advanced monitoring
      - routing maintenane
      - backtrack: restore data at any point of time without backups :warning:
    - - similar to RDS because it uses same engines
      - Encryption at rest with KMS
      - auto backups, snapshots and replicas are also encrypted
      - in fliught enc using SSL (same as MYsql or postgres)
      - possiblity to auth with IAM Token (same as RDS)
      - you are responsible for protecting the instance with SG :warning:
      - You cant ssh
    - - autoamted db instansiation and auto scaling based on actual usage
      - good for infrequent, intermittent or unpredictable workloads
      - no capacity planning needed
      - pay per second, can be most cost-effective
    - - in case HA for write node - immediate failover
      - every node does r/w vs promoting a RR as :warning: the new master
    - - cross region RR
        
        useful for disaster recovery
        
        simple to put in place
      - global DB (recommended)
        
        1 primary region (R/W)
        
        up t o5 secondary read only regions, replication lag is less than 1second
        
        up to 16 RR per secondary reagions
        
        help for decreasing latency
        
        promoting another region (for DR) has an RTO recovery time objective of < 1 minute
    - - enables to add ML based predictions to apps via SQL
      - simple, optimized, and secure integration between aurora and AWS ML services
      - supports Amazon SageMaker (use with any ML model)
        Amazon Comprehend (for sentiment analysis)
      - you don't need to have ML experience
      - use case: fraud detection, ads targeting, sentiment analysis, product recomendations
  - - - App queries elasticache, if not available (cache miss) get from RDS and store in cache else "cache hit" and retrieve from Elasticache
      - e.g. for storing user session
    - - Redis Multi AZ with auto failover
        read replicas scale reads and have high availability
        data durability using AOF
        backup dn restore features :warning:
      - memcached
        multi-node for partitioning of data (sharding)
        no high availability (replication)
        non persistent
        no backup and restore
        multi-threaded architecture : :warning:
    - - Redis AUTH
        you can set a "password/token" when you create a redis cluster
        this is an extra level of security for your cache (on top of SG)
        support SSL in flight encryption
      - Memcached
        supports SASL based auth (advanced)
    - - Lazy loading: all the read data is cached, data can become stale in cache
      - Write through adds or update data in the cache when written to a DB (no stale data)
      - Session store store temp session data in a cache (using TTL features)
    - - Gaming leaderboards are computationally complex
      - redis sorted sets guarantee both uniqueness and element ordering
      - each time a new element added, its ranked in real time, then addec in correct order