Please enable JavaScript.

Coggle requires JavaScript to display documents.

AWS - Coggle Diagram

- - - - dynamic C - HTTP cookies
    - - edge location (EL)
        
        ~region in AWS, where D is cached
        
        cache may be manipulated by developer!
      - origin
        
        where D to be distributed actually resides (S3, Ec2, etc)
      - distribution
        
        name of a CDN, set of EL
        
        types
        
        Web (web pages) <4Kb
        
        RTMP (for streaming media)
    - - URL to secure some resource (file in S3, etc)
      - only auth users can access the R
      - 1 R = 1 URL
      - N R = 1 cookie
      - features
        
        caching
        
        filtering by IP, date, path ,etc
        
        allows access to path, NO MATTER the origin
      - notes
        
        key-pair managed solely by root
        
        requires proxy app: client - app - CF
        
        signed cookies
        
        control who can access your content when you don't want to change your current URLs or when you want to provide access to multiple restricted files
      - vs pre-signed S3 URL
        
        PS URL
        
        issue a request AS A PERSON who signed the URL
        
        limited lifetime
        
        uses IAM key of the signing principal
    - - SSL
        
        SNI
        
        server name indication
        
        solves the problem of multiple SSL certificates onto 1 web-server
        
        client specifies which host it connects to, choosing the right SSL certificate
        
        not every browser supports
        
        notes
        
        requires client to specify hostname of the server (in the first handshake)
        
        TLS is a modern standard
        
        think of protection against man-in-the-middle attacks
        
        use HTTPS only
        
        use DNS with DNSSEC
        
        on aws
        
        with Route 53
        
        with custom DNS server on EC2
      - Behaviors allow to define different origins depending on the URL path.
      - set TTL to control refresh rate of content from the origin
      - handles dynamic content (API acceleration, dynamic site delivery)
      - vs S3 cross-region replication
        
        CF
        
        works globally
        
        for static content
        
        files update via TTL
        
        CRR
        
        for dynamic content at low-latency in a FEW regions
        
        must explicitly define regions
        
        files update in real-time
      - max cache hit rate
        
        separate dynamic - static content
        
        S3 - static: no headers / session caching rules
        
        EC2 / ELB - dynamic
      - vs API GW edge
        
        solutions
        
        client - edge - API GW edge
        
        client - CF (regional) - API GW edge (regional)
        
        client - CF (with caching at the edge) - API GW (regional, optional caching)
      - HTTPS & Host config cases
        
        2
        
        Client: GET https://www.example.com
        
        CF
        
        hostname: www.exaple.com
        
        SSL cert: www.example.com
        
        origin: origin.example.com
        
        origin (ALB)
        
        hostname: origin.example.com
        
        SSL cert: www.example.com
        
        if
        
        host header is forwarded (from CF to ALB)
        
        www.example.com matches SSL cert for www.example.com
        
        CF accepts
        
        Lambda Edge could set Host
        
        host header is NOT forwarded
        
        origin.example.com NOT matches SSL cert for www.example.com
        
        CF fails
        
        1 cert to MANAGE
        
        1
        
        Client: GET https://www.example.com
        
        CF
        
        hostname: www.exaple.com
        
        SSL cert: www.example.com
        
        origin: origin.example.com
        
        origin (ALB)
        
        hostname: origin.example.com
        
        SSL cert: origin.example.com
        
        if
        
        host header is forwarded
        
        www.example.com NOT matches origin example.com
        
        CF fails
        
        host header is NOT forwarded
        
        CF adds a Host header value of the origin: origin.example.com
        
        CF accepts
        
        2 certs to MANAGE
      - Path patterns are always case-sensitive
    - - delete the origin file and wait TTL
      - from AWS console
      - via API
      - 3rd party tools
    - - content caching with TTL
        
        based on
        
        headers
        
        whitelisting headers
        
        less headers -> HIGHER cache rate
        
        session cookies
        
        query string params
        
        where
        
        at EACH edge location
        
        TTL control
        
        can be set
        
        Expires / Cache-control headers
        
        0s - 1 year
      - protection
        
        DDoS
        
        integration with Shield, WAF
      - origins
        
        S3
        
        bucket
        
        static website
        
        security
        
        OAI
        
        origin access identity
        
        used for sharing private S3 objects via CloudFront
        
        bucket policy
        
        AWS Media Services
        
        live streaming / on-demand
        
        custom
        
        EC2 / ELB
        
        must allow public IP OR edge locations
        
        must have public IP!
        
        to restrict access
        
        add custom HTTP header to CF's request and parse it with ELB (ALB)
        
        secure this header
        
        API GW
        
        API GW edge
        
        on-prem
      - geo-restriction
        
        allow / block lists of countries to access content
        
        country IP is determined via 3rd party Geo-IP DB
      - edge customization
        
        Lambda Edge
        
        stored at the REGIONAL edge cache
        
        features
        
        NodeJS / Python
        
        thousands of req / s
        
        VM-based isolation
        
        features of Lambda funcs
        
        longer execution, net access, etc
        
        cases
        
        change Viewer / Origin request / response
        
        need access to net / request body / 3rd party svcs
        
        example
        
        rewrite request by a mobile device to send smaller resolution img from S3
        
        redirect based on HTTP header
        
        notes
        
        deployed in 1 region and replicated globally
        
        no free tier!
        
        auth via Cognito
        
        CF functions
        
        stored at the EDGE
        
        features
        
        written in Javascript
        
        ~ms startup
        
        millions of req / s
        
        cases
        
        high-scale, latency-sensitive
        
        change Viewer request / response
        
        URL rewrites / redirects
        
        request authentication / authorization
        
        process-based isolation
        
        2 Mb RAM / 10Kb package size
        
        1/6 cost of Lambda Edge
        
        can't use both for Viewer req / response simultaneously!
  - - - no public Internet access!!!
    - - high IO workloads
      - stable, reliable, secure connection
      - a "big pipe" from on-prem to AWS
    - - DX with VPN -> IPsec encr over private lines
        
        uses public VIF
        
        uses a public IP address to access all AWS public services
        
        w/o exposing to the Internet!
    - - is not HA by default!
      - device must support 802.1Q VLANs
      - to achieve HA
        
        dual DC
        
        create another VPN
        
        configure a dynamic route across DCs favoring the path to another DC
      - no IPsec!
      - _>price than VPN solution
    - - to handle multiple Direct Connect private virtual interfaces
        
        cross-A / cross-R
      - can connect DX and Transit GW
    - - virtual interface
      - types
        
        public
        
        connects to public endpoiints
        
        private
        
        connects to VPC res
        
        except: VPC endpoint
        
        transit
        
        connects to VPC with Transit GW
    - - dedicated
        
        1/10/100 GBs
        
        physical ethernet port for the customer
      - hosted
        
        50/500/10k MBs
        
        capacity MAY change
    - - unify <4 DXs into 1 logical
      - limits
        
        all DX must
        
        be dedicated
        
        use same bandwidth
        
        terminate at the same AWS DX endpoint (VGW)
  - - - CNAME
        
        name1 -> name2
        
        but!
        
        fails if there is name3 -> name2
        
        can't be used for naked domain names (zone apex record, like: example.com. Only www.example.com)
        
        req A/AAAA record to be used for target
      - Alias
        
        name1 -> name2
        
        but!
        
        can coexist with other aliases like name3 -> name2
        
        can map to zone apex (root) records
        
        on AWS: name2 must end with amazonaws.com
        
        cannot set to EC2 DNS name!!!
        
        TTL not mandatory!
      - MX
      - PTR
      - SOA
      - NS
        
        name servers
        
        control how traffic is routed to a domain
      - A
        
        name -> IP
        
        only IPv4
        
        AAAA
        
        same as A, but for IPv6
      - TLSA
        
        not supported by Route 53
      - Zone apex
        
        a DNS record at the root of a DNS zone
        
        example
        
        example.com
        
        must be A record
    - - domain name can be bought
      - transaction may take <3 days
      - The IP address of the DNS in a VPC is always the base of the subnet range plus two
        
        note
        
        router - net address + 1
      - mask size: /16-/28
  - - - simple
        
        multi-valued response? client picks random IP for target DNS
        
        no health-checks
      - weighted
        
        split traffic % between IPs
        
        multivalue answer
        
        allows to put health checks on each record set
        
        no health checks!
      - latency-based
        
        implies ELB
      - failover
        
        primary and backup routes
      - geo
        
        location
        
        if you need to guarantee that location is the deciding factor for user redirection
        
        provide default route!
        
        proximity
        
        complex routing with manual route setups for users / resources
        
        requires Traffic Flow usage
        
        'proximity bias'
        
        0 => region is more likely to be selected
      - multi-value
        
        1 IP - N servers (but <8)
        
        distribute in round-robin
    - - add OR / AND / NOT logic (ex: website maintenance)
      - can be setup to fail / pass based on first 5120 bytes of a response
      - for private HZ
        
        create CW metric + alarm -> point health checks to it
      - scenario
        
        replicated RDS
        
        create health-checks
        
        CW -> Lambda -> promote replica to primary AND update DNS in Route 53
    - - ELBs don't have pre-defined IP addresses (and use DNS)
      - if health check is set: unhealthy policies are removed from routing (until become healty)
      - is a global resource!
      - adding a hosted zone incurs noticeable charges!
      - domains can be automatically renewed
      - external domains can be requested to change their registered sources to Route 53
        
        use Route 53 as DNS provider
        
        update NS records where DNS was purchased
      - no Sticky Sessions!
      - allows to route between alias records based on latency
      - can't handle dynamic and static configs simultaneously
    - - traffic flow integration
        
        to build complex redirection policies
        
        versions topologies!
      - Hybrid DNS
        
        resolve DNS queries btw VPC / peered VPC / on-prem network (must be connected with Direct Connect / VPN)
        
        Resolver Endpoints
        
        inbound
        
        forward DNS Q to Route 53 resolvers (to resolve AWS DNS (ex: EC2))
        
        scenario
        
        on-prem net / private HZ
        
        outbound
        
        forward DNS Q to DNS resolvers with Resolver Rules
        
        notes
        
        not HA by def
        
        Resolver Rules
        
        system
        
        override behavior defined in forwarding rules
        
        forwarding
        
        forward DNS Q for a domain to target IPs
        
        auto-defined
        
        how DNS Q for selected domains are resolved
    - - public
        
        domain routing on the Internet
        
        DNSSEC support is only for public HZ
      - private
        
        domain routing within 1 or more VPCs
        
        must enableDnsHostnames and enableDnsSupport in VPCs
      - a container for records that define how to route T to a domain and subdomains
  - - - virtual private cloud - set of instances in AWS data center that don't have access to the internet and only private IPs
    - - virtual private net
        
        a way to connect VPC and data center
    - - Gateway to VPC -> Router -> Router Table -> network ACL (access control list) -> some VPC subnet & security group
    - - default VPC gives internet access by default!
      - 1 subnet = 1 AZ
      - security group is stateFULL
        ACL is stateLESS (rules for outbound T apply for responses to inbound T)
        
        SG allows to set ONLY allow rules
      - CIDR
        
        Classless Inter-Domain Routing (CIDR) block basically is a method for allocating IP addresses and IP routing
        
        cannot
        
        add another CIDR IP range to an existing subnet
        
        modify VPC's CIDR
      - for provisioned 256 IPs subnet - you have 251 for use and 5 reserved by AWS (network, broadcast, router, DNS - first 4 IPs, 1 IP for 'future use' - prev to the broadcast IP)
      - 1 VPC - 1 Gateway
      - 1 ACL - N subnets
        1 SN - 1 ACL
      - 0.0.0.0 route - to access public Internet
      - custom ACL blocks ALL traffic by default
        VPC default ACL allows ALL traffic by default
      - by default, VPC has: route table, security group, ACL
      - network ACL - VPC's firewall
        security group - instance firewall
      - An Elastic IP address is a STATIC public IPv4 address, which is reachable from the internet.
        
        may be used for SEAMLESS (for client) replacement of instances
      - Egress internet gateways are only for IPv6
        
        instances are UNreachable! only egress communication
      - can change the instance tenancy attribute of a VPC from dedicated to default
      - public subnet = the one that connects to the internet gateway
      - no broadcast / multicast IP!
      - Jumbo Frames - a way to increase MTU from default 1500 to 9001
        
        mind the packet fragmentation latency!
        
        reduce the TCP/IP overhead of the network traffic.
      - instance tenancy may be MODIFIED from dedicated to default for NEW INSTANCES (old not affected)
      - <125 VPC peering connections
      - more specific route prevails
        
        192,168,0.1/24 BEATS 0.0.0.0/0
    - - notes
        
        not transitive
        
        if A, B in A-B-C are peered, NO access btw A and C
        
        A,B must not have overlapping CIDRs
      - only routes traffic between source and destination VPCs.
        does not support edge to edge routing
      - features
        
        can do cross-A, cross-R
        
        longest prefix match
        (longest rule)
        
        in A - B,C setting
        
        B,C have IPs 10.0.0.0/24
        
        RT in A may have:
        10.0.0.1/32 - connects to B (longest prefix)
        10.0.0.0/24 - connects to C
        
        static route prevails dynamic (propagated) (BGP)
    - - instance
        
        manually provisioned EC2s that act as a G for a subnet
        
        must have disabled check source/destination for routing
        
        allows to detach EIP
        
        no failover, cheap
      - gateway
        
        scalable way to add a gateway to a subnet
        
        scales to <45 GBs/sec
        
        redundant within 1 AZ
        
        doesn't use security groups
        
        for failover - must manually deploy in N AZs
        
        has an EIP
        
        add it to whitelist of your firewall
      - notes
        
        must live in public subnet!
        
        translate traffic from private IPs to a single public IP
        
        i.e. connect private SN to the internet
        
        no IPv6!
        
        requires 0.0.0.0 to be redirected to NAT
        
        no VPC peering possible!
        
        cannot access NAT from a EC2 in a VPC peered SN
      - NAT gateway
        
        more powerful alternative
        
        features
        
        HA
        
        managed
        
        no security groups / bastion host
        
        runs in public internet
        
        priced 24/7!
    - - S to monitor IP traffic in subnets
      - is not cross-account
      - not monitored
        
        DHCP
        
        reserved IPs of the VPC default router
        
        traffic for instance metadata
        
        Windows license requests
        
        T to AWS DNS
        
        but! monitored to your custom DNS if its used
      - created for
        
        VPC
        
        subnets
        
        net interfaces
    - - connect VPC with other AWS svc without traffic leaving AWS
      - types
        
        interface
        
        an entrypoint for a specific AWS svc with a ENI private IP
        
        controlled by Security Groups
        
        can be shared with DirectConnect / Site-to-site VPN
        
        uses DNS names to redirect traffic
        
        notes
        
        DNS resolution must be enabled
        
        gateway
        
        features
        
        private netw with AWS services
        
        ONLY S3 / DynamoDB
        
        1VGW - 1 VPC. No peering, sharing ,etc
        
        notes
        
        controlled by VPC endpoint policies
        
        uses prefix lists in RT to redirect traffic
        
        controlled by VPC E policies
      - in s3
        
        common (price-effective) way to expose s3 objects to a VPN
        
        features
        
        pay-for-use
        
        notes
        
        bucket policy doesn't control VPC endpoints!
        
        1 VPC E connects to any number of buckets by default!
      - notes
        
        Endpoint policies - to control who can access what svc via an E
        
        aws:sourceVpc - control E access to an S3 bucket
        
        to manage access from multiple endpoints
        
        aws:sourceIp not applies! the IPs of VPCs are PRIVATE
        
        aws:sourceVpce
        
        restrict access to ONLY 1 E
        
        uses DNS!
    - - set of 'allow' rules, operating on the instance level
      - all that is not allowed - denied
      - added by default
      - ALL outbound T is allowed by default!
      - attached to ENI
      - <5 per instance
    - - features
        
        IPv4/v6
        
        NAT for EC2s with public IPs only!
        
        doesn't support private IPs only!
        
        provide route table traffic for internet-bound
        
        HA, autoscaling
      - acts as a NAT for I with public IPv4/v6
  - - - application
        
        requires >2 subnets
        
        HTTP(s) balancing
        
        to secure: add NACL before LB! origin IPs are lost after app LB
        
        features
        
        don't support EIP
        
        dynamic host port mapping
        
        multiple tasks from the same service are allowed for each container instance
        
        supports path-, host-based routing
        
        SNI (Server Name Indication) support
        
        support for >1 SSL certificates
        
        sticky sessions
        
        when client is redirected to 1 instance based on initial pick
        
        can define security group
        
        limitations
        
        targets - only private IPs
        
        400ms latency
      - network
        
        TCP / UDP (secure) traffic! for extreme performance & handling T spikes
        
        features
        
        support static IP addresses
        
        cross-zone LB
        
        can serve as Authentication and API endpoints
        
        ~100ms latency
        
        supports EIP
        
        UC
        
        whitelisting IPs
        
        1 static IP (configurable!) / AZ
        
        notes
        
        no SG! this is a transparent net device (if IP=instance id. If real NLB IP (only TCP/TLS cases!!!) - not transparent)
        
        EC2 sees the incoming client IPs
        
        targets - only private IPs
        
        cross-zone LB is not enabled by def + not free
        
        'flow hash' req routing
        
        redirect based on
        
        protocol
        
        IP
        
        Port
        
        TCP seq number
        
        cannot define security group
      - classic
        
        (legacy)
        
        have some of two others' features
        
        sticky sessions using application-generated cookies
        
        errors
        
        504
        
        app has failed (not LB!)
        
        app not responding in the timeout range
        
        for very simple routing
        
        features
        
        SAN (Storage Area Network) support
        
        listener must be configured as TCP instead of HTTPS
        
        notes
        
        doesn't support SNI
        
        should be used for a VPC endpoint service
      - gateway
        
        features
        
        operates at layer 3
        
        combines
        
        LB
        
        transparent net GW
        
        single entry / exit for all traffic
        
        uses GENEVE protocol on 6081 port
        
        manage fleet of 3rd party net (security) virtual appliances
        
        notes
        
        targets - only private IPs
    - - X-forward-for
        
        header that allows to store IPv4 of the user that made request via LB
      - LB doesn't expose its IP - only DNS name
      - has a Target Group feature - set of instances to balance
        
        The target type of existing target groups cannot be changed from "instance" to "IP"
        
        A domain name cannot be registered as a target in the target group
      - If an ELB is not terminating SSL, its listener protocol has to be TCP.
      - cross-zone LB
        
        when splitting traffic, division is calc based on overall #instances
    - - sticky sessions
        
        when user is constantly redirected to the machine that served initial request
      - cross-zone balancing
        
        LB that is aware of instances in other LB's zone and
        may send part of the load from its own zone to the other Z
      - path patterns
        
        send some kinds of T to another Z based on request URL
        
        example
        
        two zones: 1 - website, 2 - image data
  - - - provisions 2 static IPs OR you may use your OWN
      - client affinity
      - possible to control traffic via traffic dial
      - weighted traffic control
      - <1min failover
      - does health checks
    - - listens for traffic on edge locations
    - - client IP preservation NOT for NLB and EIPs
      - fits for
        
        non-HTTP cases: gaming, IoT (MQTT), VoIP
        
        HTTP cases with static IPs / fast failover
    - - DDoS protection (AWS Shield)
  - - - harder-to-manage alternatives
        
        VPC peering
        
        exposing a VPC to the public Internet
    - - NLB on the source VPC
      - ENI on the destination VPC
      - both communicating entities must have an AWS account
      - fault tolerance? NLB and ENI must be BOTH in
        multiple-AZs
    - - can be cross-A
      - no peering, routing, etc
  - - - works on hub-and-smoke model
        
        this model minimizes management overhead
      - works with DirectConnect, VPC connections
      - (uniquely) supports IP multicast
        
        I in VPCs can access NAT GW, NLB, PrivateLink, EFS in OTHER attached VPCs
      - can be cross-R
      - route tables: limit VPC-VPC communication
      - can peer cross-R
      - share cross-A with Resource Access Manager (RAM)
    - - many TGWs = hub
  - - - low-cost, easy to manage
      - hub-and-spoke model
      - operates over public Internet, but all traffic is encrypted
    - - connect remote locations with AWS / each other for low-intensive workloads
      - failover btw on-prem nets
  - - - ingress traffic
      - traffic within 1 AZ in a VPC
    - - inter-VPC (inter-region) nets communication (over public IPs)
        
        costs 2X than over private IPs
      - traffic from VPC to public net
      - traffic btw VPC's AZs
  - - - static routes support
      - BGP (Border GW protocol) peering & routing
        
        BGP
        
        dynamic routing. share routes automatically (eBGP)
    - - depends on available internet connection
    - - connect customer and virtual private gateways
    - - 2 VPN tunnels for HA
    - - option set
        
        to customize params issued to clients upon a DHCP req
        
        E: set an NTP server to custom
  - - - setup bandwidth, redundancy, etc
    - - no managed alternative on AWS
  - - - connect N VPCs with each other / on-prem using N VPNs
    - - allows overlapping CIDRs, net-level packet filtering
      - complex routing rules
  - - - IPsec encr
      - routing
        
        static
        
        must create RT manually
        
        dynamic
        
        use BGP
        
        specify ASN (autonomous system number) of on-prem GW and VGW
    - - on-prem VPN appliance with public IP + GW
      - private VPC + VGW
      - on-prem GW connect to VGW over (2 - for redundancy) VPN tunnels (site-to-site VPNs)
    - - accelerated with GlobalAccelerator - for worldwide nets
      - to connect on-prem users to the internet
        
        setup NAT instance
        
        NAT GW won't work due to its restrictions
        
        can customize what to allow
        
        setup on-prem NAT
      - for small cases - ClientVPN
        
        connect from private net on-prem / AWS to a user's computer
        
        uses OpenVPN
      - best practices
        
        N VPCs, 1 on-prem: separate VPN for each VPC - on-prem GW
        
        or Direct connect GW
        
        shared services VPC
        
        customer - (site2site) - VPC A - (peering) - VPC B, C, D
        
        share apps from on-prem with AWS
      - failover
        
        active-active S2S
        
        2 on-prem nets connected with each other AND over 2 tunnels with VGW
        
        1 tunnel fails - recovery via the other one
      - will typically use the public internet and therefore cannot offer consistent network performance
- - - - each object has
        
        key
        
        value
        
        version iD
        
        metadata
        
        subresources
        
        access control list
        
        who can do what on an obj
        
        torrent
        
        Entity tag (ETag)
        
        with MD5 hash of the object
        
        not for multipart uploads!
    - - R after W consistency for PUTS of new obj
      - eventual C for overwrite PUTS and DELETES
    - - IAM policies (user-based)
      - policies on concrete obj, bucket (ACLs)
    - - when retrieving, user targets closest zone's cloudFront which then targets storage (more efficient)
    - - allows to purge / transition obj to different storage C based on some timeline
      - may be used in conjunction with versioning
    - - prohibit to delete / upd O
      - modes
        
        compliance
        
        O version can't be modified by ANY user for the duration of retention period
        
        governance
        
        same as compliance, but U with P can modify the O
      - legal hold
        
        prevents V from modification (until removed manually)
      - retention period - time till the R applies
      - Glacier has its own Vault Lock (with the above options)
      - enforce WORM (write once - read many) semantics on obj
        
        upon updates, new obj version is created and the original obj is not modified
    - - makes reads up to 400% faster compared to filtering O by hand
      - using SQL, filter out objects of interest
    - - types
        
        cross-region replication
        
        not default!
        
        same region
      - not enabled by default
      - copy of an object won't have the same R policy
      - version O in both buckets on copy!
      - delete markers are NOT R
      - MUST have versioning enabled
    - - uploading not to S3 directly, but to a S3's zone bsed on CloudFront Edge Network
      - almost always ~100% faster than usual S3 upload
      - pay only for transfers that are accelerated
    - - an agent that syncs data between on-prem / S3 / EFS / FSx and S3 / EFS / FSx for FULL MIGRATION purposes
      - features
        
        works for any file size / any FS type
        
        secure
        
        can copy over Direct Connect OR public Internet
      - features
        
        TLS encr
      - alternatives
        
        Transfer Family
        
        transfer to S3 / EFS with FTP(s)
        
        UC
        
        sharing files, CRM, public data
        
        endpoints
        
        public
        
        temp IP
        
        VPC private
        
        static private IP
        
        NACLs & SG
        
        VPC with Internet
        
        static private, public IPs
        
        SG security
    - - static website hosting
        
        simple, massively scalable hosting
        
        Origin Access Identity cannot be used with S3 website origin (works for bucket access only)
      - tags
        
        attach tags to obj
      - events
        
        types
        
        access logs
        
        can take HOURS to propagate
        
        event notifications
        
        SECONDS to propagate
        
        CW / Lambda as destination
        
        trusted advisor
        
        is bucket public?
        
        CW
        
        must enable CT obj-level logging
        
        target downstream Lambda, etc
      - requester pays
        
        bucket owner is charged ONLY for obj storage
        
        implies that requester has AWS account!
        
        if IAM role is assumed - its owner pays
        
        scenario
        
        sharing large datasets cross-A
      - bittorrent support
    - - HTTP
        
        no encr!
      - HTTPS
        
        a must for SSE-C encr
        
        for encr in flight
  - - - less freq access, but fast when needed
      - if accessed - costs are higher than usual!
    - - ML for moving obj to the best storage class
      - more expensive than standard if frequent reads
      - 99.9% availability
    - - IA without multiple region availability
      - 99.95% avl
      - data is lost if AZ is destroyed
    - - designed to serve even if two replicas go down
      - 99.99% avl
    - - features
        
        cheap
        
        all data is AES-256 encrypted by default!
      - concepts
        
        Vault
        
        stores objects based on IAM access and Vault Lock policies
        
        Vault lock
        
        immutable
        
        extra rules like MFA, no deletes
        
        notes
        
        24 hours to confirm lock for vault
        
        archive
        
        <40Tb
      - types
        
        Instant Retrieval
        
        ~ms access, min 90 days storage
        
        Flexible (expedited) Retrieval (formerly, Glacier)
        
        types
        
        expedited
        
        1-5 mins
        
        standard
        
        3-5H
        
        bulk
        
        5-12H
        
        free
        
        min 90 days storage
        
        Deep archive
        
        types
        
        standard
        
        12H retrieval
        
        bulk
        
        48H
        
        min 180 days storage
    - - retrieval up to 12 hours
      - very cheap
    - - freq accessed, non-critical
  - - - depends on pair (internet speed, data size)
        
        ex: 100Mbs -> data should be >5Tb in size
    - - transfer from physical storage to AWS <80Tb
    - - =snowball, but with Lambda and clustering features
      - features
        
        multiple concurrent devices => higher speed
        
        has onboard S3-Compatible Endpoint
        
        seamless transfer from a data warehouse S3 interface
        
        can run EC2 / Lambda for cases on edge locations
        
        OpsHub
        
        UI for management of snow-family
        
        Adapter for snowball
        
        to increase 10x transfer rates (from 25-40 MBs)
      - types
        
        compute-optim
        
        <42TB
        
        has (optional) GPU
        
        storage-optim
        
        <80TB
        
        Snowcone
        
        <8Tb
        
        possible to use with DataSync
        
        note. TBs here are for USAGE. These devices can migrate MUCH MORE (3-100x)
    - - only RJ45, SFP, QSFP interfaces
      - multiple clients (in multiple terminals) = faster data transfer
  - - - (per prefix)
        prefix = folder in which target files reside
        
        bucket/folder/file: prefix - 'folder'
    - - allows to parallelize uploads
      - upload w/o info about final file size
      - higher throughput
      - quick recovery from net issues
      - ability to pause / resume uploads
      - keep in mind! checksums of original and uploaded objects may be compared to enforce transmission integrity
        
        use custom metadata parameter for checksum
      - embedded in CLI (aws s3api)
      - fails? set lifecycle policy to delete corrupted files
    - - improves resiliency on failures
      - to retrieve PARTIAL data (for e.g., header)
    - - <30k requests / sec
  - - - on the fly (in transit)
        
        SSL/TLS in HTTPS sessions
        
        IPSec for VPN
        
        AWS Certificate Manager
        
        manage SSL/TLS cetificates / keys
        
        features
        
        supports CloudFront / API G / LB
        
        can import existing / 3rd party certificates
        
        managed certificate renewal
        
        supports *.domain.com domains
        
        notes
        
        regional svc
        
        cannot copy cert across R
        
        to host public cert on aws
        
        upload your own via CLI
        
        must be MANUALLY renewed
        
        have CM create and manage cert for you
        
        must
        
        verify public DNS
        
        issue with a trusted CA
        
        to host private certs
        
        create via your CA -> your app must trust your private CA
      - at rest (data encryption)
        
        S3 managed keys
        
        AWS KMS (another managed keys)
        
        server-side encr with client-provided keys
      - client side
    - - user-based
        
        IAM policies
      - resource-based
        
        bucket P
        
        allows cross-A
        
        may force obj encr at upload
        
        (optional) conditions on
        
        src IP: for public / EIP (not private!)
        
        MFA
        
        CloudFront origin identity
        
        src VPC / VPC endpoint
        
        only for VPC endpoint
        
        obj ACL
        
        finer grain
        
        bucket ACL
    - - temporary! valid for 1H by default
      - for down/up-loads
      - URL users INHERIT permissions of the person who created URL
      - scenarios
        
        allow only logged-in U to download content
        
        handle download access for ever-changing list of U
    - - why
        
        to simplify complex bucket policies with lots of fine-grained rules
      - a proxy btw S3 and users
      - shared VPC access
        
        VPC AP + VPC gateway endpoint
  - - - programmatic only
    - - programmatic & console
    - - programmatic only
  - - - JSON, ORC, Parquet
    - - Q are logger in CT
        
        you can stream into CW and apply metric filter to get CW alarm
      - ready-for-use Q for VPC flow logs, CT, Cost and Usage, etc
  - - - recommended use of EventBridge
  - - - VM that runs on top of some machine on-prem
    - - File (NFS, SMB)
        
        VM bridge btw on-prem NFS and S3
        
        features
        
        caching of recently used D
        
        NFS / SMB protocols
        
        metadata and dir structure are preserved
        
        can be mounted on MANY servers
        
        RefreshCache API to control backup restores
        
        solutions
        
        migration
        
        on-prem - FGW1 - S3 - FGW2 - EC2s in VPC
        
        fast replication (read replicas)
        
        on-prem1 - FGW1 - S3 - FGW2 - on-prem2
      - Volume (iSCSI protocol)
        
        captures only CHANGED blocks
        
        1Gb - 16TB
        
        types
        
        Stored
        
        mounted on-prem
        
        scheduled replication of full on-prem datasets to S3
        
        <16T
        
        cached
        
        actual data is stored in S3! some of it cached on-prem for low-latency access
        
        <32T
        
        the only way to access on S3 the volume - create EBS snapshot
      - Tape
        
        used for archiving using physical tapes
        
        use-case
        
        can initially be installed in the on-premises environment utilizing the existing enterprise backup product to start the transition without losing access to the existing backups and archives. Over the duration of the migration, most (if not all) the backup cycles will be replaced by the new VTL & VTS tapes
        
        notes
        
        restore only the tape ENTIRELY, not a single file
      - FSx file GW
        
        features
        
        Windows compatibility
        
        SMB, NTFS, Active directory, etc
        
        local cache for freq accessed data
        
        integration with AWS FSx File server
    - - used in cloud migrations
      - no SLA!
  - - - cloud front - S3
    - - S3 - Lambda - Dynamo
    - - separate dynamic & static content!
        
        for e.g., S3 - for static, EC2 - dynamic
  - - - once enabled - cannot be disabled (only suspended)
      - MFA delete
      - logs any W/DELETE
      - ANY version is an INDEPENDENT obj
        
        incurs costs for storage on itself!
- - - - price depends on memory allocation to a F
      - 16*10^-6 $ per GB/second
  - - - 1 event = 1 F
    - - src events are BATCHED and then sent to L
    - - async sink to location based on processing status
        
        success
        
        failure
        
        use instead (recommended) of DLQ or together
    - - by def - $LATEST
      - immutable, non-decreasing, each V has ARN
      - V = code + config
    - - mutable pointers to L versions
      - scenario
        
        'dev' -> $LATEST, 'prod' -> v2
      - supports WEIGHTED routing
        
        can redirect traffic to other L versions
        
        with CodeDeploy
        
        assign X % of traffic to diff versions, varying X until it gets to 100%
        
        error? sets X=0%
  - - - including those with many Lambdas
  - - - Firehose, Amazon Lex, API Gateway
    - - S3, etc
  - - - code is too large? Break it maximally down
        
        doesn't help?
        
        Beanstalk / ECS / mount EFS
    - - sync
        
        6Mb
      - async
        
        256Kb
    - - cold invoke
        
        ~100ms
      - warm invoke
        
        ~ms
      - provisioned concurrency
        
        to reduce #cold starts
      - latency of downstream components
        
        API Gateway / Cloud Front invocation - 100ms
      - monitor the whole architecture with X-ray!
  - - - Reference the function using an unqualified ARN
      - Reference the function using a qualified ARN and the $LATEST suffix
    - - must implement Lambda Runtime API
    - - write CW logs
      - (optional) be used with X-Ray
    - - sync
        
        results return right away
        
        error handling on the client side
        
        SDK / CLI / API GW
      - async
        
        S3, SNS, CW events, etc
        
        3 retries on errors
        
        ensure idempotency!
    - - consider creating a Lambda Layer for the reusable code
        
        a ZIP archive that contains libraries, a custom runtime, or other dependencies
  - - - scheduled CW events + Lambda
  - - - allow L do smth
    - - allow other accounts / svc to use L
  - - - by default - publicly accessible
    - - expose to internet via
        
        NAT + IGW
      - allow to talk to other svcs not in this subnet
        
        use VPC endpoint
        
        Cloud Watch logs work w/o gateways / endpoints!
- - - - snapshots are sets of volume's blocks. They are INCREMENTAL
      - for root device EBS - you must stop the instance first
      - AMIs are copied with empty configs and you need to manually reinstall all configs from the origin AMI
    - - data survives only reboots
      - compared to EBS, which allows to reboot, whatever
      - fastest disk IO
      - not resizable, backups are manual
    - - OS boots
        
        bootstrap scripts run (bash scripts to configure instance env, etc)
        
        services start (Apache, etc)
    - - alternative - using SMS
    - - amazon machine image
      - box apps that are long to install into a box and use
  - - - options to TURN ON: termination protection, encryption, delete storage volume on termination of an instance
      - inbound traffic is blocked by Def
    - - inbound (ingress) rules
      - changes to security groups take effect immediately
      - security group - set of In/out rules
      - SG does NOT allow to block IPs! Use Net access Lists for this
      - no deny rules for SG
      - SGs are STATEFUL (in/out rules are separate)
    - - elastic FS (NFS) for use by several EC2s
      - must be mounted explicitly to EC2 I
      - RaW consistency
      - Windows FS for windows!
      - FS Lustre for HPC and ML!
        
        stores data directly on S3
      - MultiAttach: <16 I can have the volume
      - features
        
        pay-for-use (unlike EBS)
        
        multi-AZ
        
        configure mount targets in >=1 AZs
        
        (not mount shares!)
        
        can be mounted on-prem
        
        not recommended
        
        rather, use DataSync btw on-prem and AWS-hosted EFS
        
        encryption
        
        in-transit
        
        when mouting explicitly add a TLS option "-o tls"
        
        no CMK required
        
        at rest
        
        KMS. a CMK is required
        
        notes
        
        not enabled by default!
        
        autoscaling!
      - pricing
        
        3x more than EBS
        
        20x more than S3
      - notes
        
        supports only Linux instances!
        
        pay-per-GB
        
        uses security groups for A control
        
        may be used in 1 VPC only
        
        and create 1 mount target (ENI) / AZ
        
        IPv4 only! not DNS
        
        access points (~S3 AP)
        
        file-system policy
        
        ~ resource policy
        
        by def - full access
      - classes
        
        Scale
        
        10Gb/s throughput
        
        PB-scaling
        
        Performance
        
        general-purpose, latency-sensitive cases
        
        max IO
        
        Throughput
        
        set throughput regardless of storage size
        
        bursting
        
        storage classes
        
        storage tiers
        
        std
        
        freq accessed files
        
        infrequent access
        
        lower cost to store / higher to retrieve
        
        lifecycle policy to change
        
        availability & durability
        
        regional
        
        for prod
        
        one-zone
        
        for dev
        
        backup enabled by def
        
        compatible with EFS one-zone IA
    - - run any DB with full control in your hands
      - option when there is no AWS DB alternative / no migration possible
    - - shared (default)
        
        Multiple AWS accounts may share the same physical hardware
      - dedicated host
      - dedicated instance
    - - R
        
        memory-opt
      - C
        
        CPU-opt
      - M
        
        balanced EC2
      - I
        
        high IO
      - G
        
        gpu support
      - T2/T3
        
        burstable (up to a capacity)
        
        unlimited burst
  - - - fixed rate / hour
    - - bid whatever price you want for compute / storage
      - if used for <1 hour and terminated by
        
        AWS
        
        price not charged
        
        yourself
        
        priced!
    - - capacity reservation + discounts
      - can be selled if not needed!
    - - physical EC2 server dedicated for use
      - features
        
        control over which instances are deployed on this host
        
        can run only 1 type of EC2
        
        host affinity
        
        EC2 reboot keeps the same host
      - notes
        
        aren't used in auto-scaling
        
        price is higher than that of the standard EC2
    - - one of: on-demand / spot / reserved
      - pricing
        
        +2$ / hour
      - this is a VM on some real hardware
    - - discount based on commitment on long-term usage of EC2s
      - types
        
        instance
        
        <72% savings (~Standard RI)
        
        flexible in terms of I size, tenancy (dedicated / default), OS
        
        limits usage to an instance family within a region
        
        compute
        
        <66% (~Convertible RIs)
        
        flexible in I families, regions, compute types (Fargate, ECS, etc)
        
        SageMaker
        
        <64%
        
        for SM workloads
      - notes
        
        beyond SP? price as for on-demand
  - - - has spot block option
      - up to 90% price drop
      - proactive rebalancing
      - (optional) capacity-optimized allocation strategy
    - - price is AZ-dependent! some AZs are more volatile than others
    - - mix of spot I and on-demand I
      - set max price for spot instances in the fleet
      - mix of I types
      - spot I pools
        
        lowestPrice
        
        diversified
        
        great for availability, long workloads
        
        capacityOptimized
        
        choose pool based on #I
    - - AWS notifies 120s before shutdown which will certainly happen
      - AWS may shutdown an I outward the bought plan. In this case usage is not charged
      - use only idempotent request handling
      - to stop incurring costs
        
        terminate
        
        Spot request
        
        running Spot instances
  - - - elastic block storage
      - a physical block storage that replicates across AZ
      - "virtual volume"
    - - Magnetic
      - general-purpose (SSD)
        
        16k I/Ops max
      - gp2 / io1 / io2 / io2 Block Express
        
        16k / 64k / 128k / 256k I/Ops
        
        mission critical / high-throughput workloads
        
        can be used as boot vol
      - High throughput HDD
        
        500 IOPS
      - cold hard-drive
        
        good fit for large, sequential cold-data workloads.
    - - encrypted can be converted only to encrypted
      - possible to create encrypted from unencrypted! recreate from a snapshot
      - deliver performance for workloads that require the lowest-latency access to data from a single EC2 instance
      - single AZ
        
        must match that of an EC2
      - EBS vol cannot be copied cross-account
        
        only snapshots
        
        encrypted?
        
        copy with CMK
      - to create an encrypted root volume
        
        stop the I
        
        create AMI with 'encrypt target EBS snapshots'
        
        redeploy the instances with these AMIs
      - if restored, 1st access means the EBS must be downloaded from S3 => extra time to wait
      - instead of RAID
        
        has Multi-volume snapshots
        
        may be redundant if migrating from on-prem commodity, as EBS is reliable enough by itself
      - may use RAID0 to increase IO throughput
      - for max throughput choose EBS-optimized I
      - 1 I - N EBS
        
        but! io-family allows M I - N EBS in a single AZ
        
        choose file-system accordingly. must be cluster-aware
      - size <-> IOps
    - - snapshots
        
        backup of data
        
        incremental (block-wise)!
        
        new data = new (small) block to existing snapshot blocks
        
        notes
        
        take IO!
        
        stored in S3 (no access)
        
        can create AMI
        
        can be copied to another region
        
        restored EBS must be pre-warmed
        
        use Fast Snapshot Restore (FSR)
        
        can be automated using Data Lifecycle svc
  - - - local (one rack) set of EC2s (low latency, etc)
      - choose I-type EC2s with enhanced netw capabilities!
    - - individual EC2s in different AZs
      - <7 I per AZ
    - - multiple EC2 in one AZ but different racks
      - best for massively distributed computing (Hadoop, Cassandra, etc)
    - - to change PG
        
        stop I
        
        use CLI to move I
        
        start I
  - - - virtual net card
      - low-budget, HA
      - attach
        
        warm
        
        when EC2 is stopped
        
        cold
        
        when EC2 is being launched
        
        hot
        
        when EC2 running
      - detach
        
        ENI can be detached when the instance is running or stopped
    - - enhanced networking adapter
      - uses single-root IO virtualization for high-performance net in specific instance types
      - 10-100Gbs bandw
      - should be preferred to Virtual Function networking (<10Gbs, is legacy)
    - - elastic fabric adapter
      - powerful net device for HPC
      - how
        
        by-pass OS-level and communicate directly with net device
      - notes
        
        only Linux
        
        used in tightly coupled workloads
        
        because uses MPI
  - - - only on-demand / reserved
  - - - standard
      - convertible
        
        more flexibility: change instnace family, OS, tenancy, payment options
        
        benefit from price reductions
        
        but
        
        not for sale (not yet)
        
        has lower discounts
    - - can't change region
      - assign AZ straight away!
        
        otherwise - availability is a concern
      - can change AZ!
      - reserve capacity per zone / on-demand to ensure you have the instance types you need available when they are required
        
        regional RIs will not prevent a capacity issue within an Availability Zone
        
        for this - reserve Zonal RI!!!
  - - - instance status
        
        check EC2 VM
      - system status
        
        check EC2 hardware
    - - same
        
        private IP
        
        public IP
        
        EIP
        
        metadata
        
        placement group
- - - - security
        
        restrict based on geo
        
        use WAF for IP-based rules
    - - WAF & CF - initial filtering
        
        NACL - for geo-specific IPs
    - - notes
        
        Ephemeral ports are required to allow the server to communicate back to the client.
        
        Rules are evaluated IN ASC # ORDER
        
        As soon as a rule matches traffic, it's applied immediately
        
        max 20 rules by default
        
        may be extended upon request
      - By default,
        
        default: it allows all inbound and outbound IPv4 traffic and, if applicable, IPv6 traffic
        
        each custom network ACL denies all inbound and outbound traffic
      - subnet-level firewall for IPs
    - - features
        
        analyzes req parameters
        
        filter req based on rules
        
        can include IP addr / HTTP req / URI / geolocation / req size / req rate
        
        managed rules
        
        set of rules to protect in common scenarios
        
        types
        
        baseline
        
        use-case specific
        
        SQL, wordpress, etc
        
        IP reputation
        
        based on malicious IP list
        
        bot control
      - web application firewall
      - protects
        
        API Gateway
        
        Cloud Front
        
        ALB
        
        AppSync (GraphQL)
        
        against
        
        SQL injection
        
        cross-site scriptin (XSS)
      - notes
        
        for DDoS protection - use AWS Shield
    - - intrusion detection / prevention systems
    - - DDoS
        
        level 4 OSI
        
        SYN flood (lots of TCP req) / UDP reflection (lots of UDP req)
        
        other
        
        DNS flood / slow loris (lots of open HTTP connections / etc
        
        protection
        
        AWS Shield
        
        std
        
        free
        
        protects at >= level 4
        
        advanced
        
        24/7
        
        wide range of svcs protection
        
        WAF
        
        Cloud Front + Route 53
        
        Shield -> DDoS protection at the edge
        
        protect availability with global edge network
        
        separate static and dynamic res!
        
        static
        
        S3 / CloudFront
        
        dynamic
        
        EC2 / ELB
      - app atacks
        
        cache bursting
        
        overload DB using freq cache invalidation
        
        etc
    - - features
        
        common set of security rules
        
        WAF rules
        
        SG, ENIs for VPC
        
        Aws Shield Advanced
      - manage rules in ALL A in your Organization
  - - - encrypt data <4Kb
        
        satisfies req to encrypt 'before writing to disk'
      - pay-by-API-call
      - audit with CloudTrail
      - keys
        
        types
        
        customer managed
        
        allows key rotation (old keys are stored) every year
        
        managed via key policies
        
        create and manage keys manually
        
        use case
        
        envelope encryption
        
        AWS managed CMK
        
        free
        
        used by services
        
        auto-rotation every 3 years
        
        AWS owned CMK
        
        cross-account
        
        no way to audit yourself
        
        encryption
        
        symmetric
        
        AES-256
        
        you never get it unencrypted
        
        implies KMS API usage
        
        use case
        
        envelope encryption
        
        assymetric
        
        RSA, ECC
        
        when KMS API isn't available (outside AWS) but you need to decrypt smth
        
        public key is shared
      - multi-regional
      - integrates with all AWS services
      - Cloud HSM integration
        
        custom KMS key store cluster
        
        notes
        
        not HA by default
      - multi-region keys
        
        one key across N R
        
        not global. This translates into one primary K and N replicas
    - - use the KMS API to encrypt all data before saving it to disk
      - only SYMMETRIC encr
  - - - full-managed
      - multi-AZ
      - not HA by default
      - operates in dedicated VPC
    - - regulatory compliance requirements
  - - - serverless
      - stores any secret data
      - stores params in hierarchies (example: prod/dev tree hierarchy)
      - has version control
      - has TTL
  - - - auto key rotation
      - cheaper than Parameter Store
      - gen random secrets
      - cross-account sharing
    - - secret / month
  - - - other ways: CloudFront + WAF
    - - Standard
        
        no cost
        
        protection against level 3, 4 (OSI model) attacks
      - Advanced
        
        costly
        
        24/7 support
        
        protects more svcs
        
        only against DDos!
  - - - authentication
        
        authorization
        
        trust
        
        do others whom I trust, trust you as well?
        
        SAML-based federation, etc
        
        are you allowed to do X? IAM
        
        Proof of who are you
        
        SSL / MFA
      - who are you?
      - Root user, IAM user, temporary creds
  - - - validates user claims via IP and gives access ultimately
    - - communicates with SP and IP to gain access
    - - tells service provider that a user is valid to access svc
  - - - authentication + authorization
    - - authorization ONLY
      - issues tokens to clients
    - - authentication ONLY
  - - - shared responsibility
        
        AWS and customer are responsible for something both
        
        physical hardware infrastructure
      - customer's responsibility
        
        user authentification
        
        client-side encryption
        
        firewall configuration
        
        Identity and Access Management
- - - - 0-3600s TTL (300s - default)
      - works per-stage
      - may be invalidated with header - Cache-Control: max-age=0
      - 0.5-237Gb
    - - used for
        
        real-time apps (chatbots, etc)
        
        stateful apps (with DynamoDB, etc)
  - - - prevents cross-site scripting
      - browser permits scripts from webpage1 to access WP2 only if both WPs have the same origin
      - enforced by web-browsers!
    - - allows to request restricted WP resources (for e.g., fonts) from server in another domain
      - to relax same-origin policy
      - is a source for
        
        error "origin policy cannot be read at the remote source
        
        response "these other domains are approved to GET this URL"
    - - but! WAF can't set rate-based limit rule
    - - upload file to S3
        
        API FW -> Lambda -> return pre-signed URL for upload
  - - - dev/prod
  - - - edge-optimized
        
        req are routed through CloudFront edge
        
        Gateway is still in one region
        
        for global clients
      - regional
        
        for clients within the same region
        
        MANUALLY combine with CloudFront
        
        more control over caching, etc
      - private
        
        access from a VPC using VPC endpoint interface (ENI)
        
        res policy to control access
  - - - client errors
    - - server errors
      - 502
        
        bad gateway
        
        incompatible output from Lambda, etc
      - 503
        
        service unavailable
      - 504
        
        req timed-out
  - - - ~S3 bucket policy
    - - browser-based security
      - control which domains can call an API on API GW
    - - IAM
      - Lambda authorizer
        
        use L to verify custom SAML / OAuth / 3rd party auth
      - Cognito user pools
        
        knows how to validate tokens from Cognito
- - - - this way - using EBS is recommended O for storage
      - RAID 0 + additional EBS to improve performance
  - - - read-only replica of production DB
      - access is load balanced
      - very READ-HEAVY cases
      - requires automated backups turned ON
        
        to disable backups
        
        Remove the Read Replicas first
      - non-transactional storage engines for MySQL don't support replication!
      - replication
        
        sync
        
        multi-AZ case
        
        T instantly replicate
        
        async
        
        Read replicas case
        
        updates may be synced with delays
      - can be cross-region
    - - replicate in another AZ
      - used for disaster recovery
      - exclusively for FAILOVER! replicas don't accept requests till master goes down
        
        to create an HA architecture
      - failover 1-2 mins!
      - keep in mind!
        
        networking routes must be in sync btw master and standby
        replicas
      - cross-region solution
        
        create async read replica -> CW health checks -> Lambda -> promote replica to master -> update DNS with R53
    - - runs on VMs! (not serverless)
        
        except for Aurora Serverless
      - Automated Backups
        
        1-35 days retention, any time recovery
        
        restores to a NEW RDS endpoint
        
        live independently of original RDS
      - RDS proxy for Lambda
        
        protection against 'many Lambdas open too many connections to RDS'
        
        features
        
        autoscaling
        
        IAM auth
        
        no need to manage connection pools
    - - for Mysql, Psql
        
        but!
        
        3-5x performance
        
        lower cost
        
        no single-AZ!!!
        
        failover in <30s
      - features
        
        <64 TB
        
        scales in 10Gb chunks
        
        <15 replicas
        
        snapshots may be shared with other AWS accounts
        
        maintains 6 DB copies by default!
        
        increases in 10GB increments up to 64TB
        
        failover
        
        cross-region replication
        
        copies entire DB!
        
        global DB (recommended)
        
        1 primary (RW) and <6 secondary (R) regions
        
        replication in <1s
        
        <1min RTO
        
        decreases latency as well
        
        multi-master
        
        immediate failover for a writer node
        
        every node is RW
        
        autoscaling
        
        endpoints
        
        types
        
        reader
        
        for R
        
        custom
        
        connect to different DB instances
        
        cluster
        
        for W
        
        instance
        
        connect to a specific Aurora I
      - Aurora Serverless
        
        option for unpredictable / infrequent workflows
        
        no costs when idle
    - - provisioned IOPS over standard storage when
        
        workload is critical / higher IO is required
      - spot errors by looking for 'error node' using RDS API
      - Oracle RAC != Oracle
        
        O RAC can be deployed on VMware Cloud on AWS
        
        RAC is not supported on RDS
      - RDS Oracle backup options
        
        all, except Replication Backups
      - Modifying RDS DB Parameter Group values will require a database restart to take effect
        
        infeasible to do in response to traffic spikes
      - to defer updates
        
        defer indefinitely until they are comfortable
      - has the capability to perform this failover automatically
      - Route 53 can't simply monitor RDS health!
        
        only via CW alarm / custom API
      - general SSD storage - 100Gb. Given 3 IOPS / 1GB volume. 100Gb - only 300 IOPS
      - up to 100,000 client connections
      - IP address of the RDS instance should not be used
    - - KMS at-rest for EBS volumes / snapshots
      - transparent data encryption for Orackle / SQL server
      - can copy unencr snapshot -> encrypted
      - no CT for tracking RDS queries
      - supports IAM authentication for Mysql / Postgres
  - - - features
        
        3 replicas by default
        
        eventual / strong consistency in R
        
        eventual R = R will be consistent in some time (1 sec in DynamoDB)
        
        ms-mcs latency
        
        uses SSD
        
        Capacity
        
        on-demand
        
        flexible C, more expensive
        
        autoscaling
        
        troublesome downscaling if was upscaled
        
        to tackle throttling of requests
        
        fixed
        
        achieve ACID with DynamoDB Transactions
        
        set TTL to purge old data
        
        Global tables
        
        cross-region replication for Dynamo
        
        UC
        
        low-latency, DR
        
        active-active replication, many instances
        
        must enable Streams!
      - DynamoDB Accelerator (DAX)
        
        cache for DynamoDB
        
        features
        
        point-in-time-recovery
        
        <35 days
        
        possible to use in multi-AZ
        
        <10 nodes
        
        secure (KMS, VPC, etc is a def)
        
        notes
        
        bad for intensive W!
        
        bad for client-side caching apps
        
        solves hot-key problem (too many reads)
        
        5min TTL for cache by def
        
        use ElastiCache for more-than-simple-read access
      - notes 2
        
        doesn't support MongoDB! use DocumentDB instead
        
        DocumentDB = Mongo on AWS
        
        name & value for a record combined must be <400 Kb
        
        each R has unique primary key => hash => used for partitioning
        
        secondary indexes
        
        global
        
        partition K and sort K may be different from those in the T
        
        when
        
        fast Q of attributes outside PK
        
        can be defined after T is created
        
        local
        
        save partition key, but diff sort key
        
        when
        
        with the same PK, fast Q on some other attribute
        
        must be defined before T is created
        
        limitations
        
        takes more storage
        
        limited #indexes, #attributes per index
        
        to avoid sequential R
        
        not enough RCUs? read throttling
        
        partition and sort keys attributes must be defined as type string, number, or binary
        
        sort key - to speedup lookups
        
        PK + SK must be a unique combination
        
        to get a record you MUST identify it with some index
      - notes 1
        
        no charge for single-region secondary indexes
        
        charge for R/W and storage
        
        calc #partitions
        
        result=max(by size, by capacity)
        
        by size
        
        size / 10Gb
        
        by capacity
        
        total RCU / 3000 + total WCU / 1000
        
        RCU = read capacity unit
        
        avoid hot-spot from bad partitioning!
        
        AS global secondary indices = copy of the table
        
        support AS
        
        AWS AS
        
        inexpensive
        
        unable to automate scale-down
        
        on-demand AS
        
        when?
        
        no idea what scaling is needed
        
        for time series - create a new table per sufficiently large period
        
        best practice: min number of tables
        
        performance improvement
        
        archive data & recreate the table
      - streams
        
        ingest to Dynamo and, with extra processing (EC2, Lambda, Kinesis sink), to a data lake
        
        24H retention
        
        react to changes in T in real-time
        
        vs Kinesis Streams?
        
        when TTL must be >24H
        
        when further Kinesis Firehose / Analytics sinks are required
  - - - can migrate from almost anything to almost anything (including on-prem, EC2, etc)
    - - heterogeneous DBs
        
        SCT (schema conversion tool) may be used to convert btw schemas
        
        no Couchbase support
        
        not used for same DB engines
        
        with Snowball
        
        export data with SCT to a device
        
        ship device to AWS
      - homogeneous DBs
        
        no SCT!
    - - during migration, origin remains fully operational
      - can't replicate ElasticSearch
    - - continuous replication
      - src DB remains fully operational
  - - - snapshots
        
        created automatically
        
        every 8hours / 5GB / scheduled
        
        retained until deletion
        
        may be copied to another region
        
        if KMS-encrypted
        
        transfer key to the target region
        
        transfer snapshot
        
        can restore snapshot in a NEW CLUSTER
        
        point-in-time, incremental
      - WLM
        
        workload manager
        
        types
        
        automatic
        
        manual
        
        manage Q priorities
        
        3 queues: superuser / short-running / long-running
      - concurrency scaling
        
        scale based on current #users
        
        pay-per-second
    - - compute node hours
      - backups
      - data transfer
    - - read data from S3
      - can join data from S3 and other ext. sources
      - notes
        
        must have a Redshift cluster available!
    - - no multi-AZ! best option - multi-node cluster
        
        data replication
        
        node recovery
      - single-node C doesn't support replication
      - UPSERT - func to update if PK exists OR insert otherwise
      - sporadic Q? Use Athena
      - node types: leader (Q planning) & compute
  - - - Memcached
        
        is VERY fast
        
        allows to operate in multi-threaded mode
        
        no encryption!
        
        no multi-AZ!
        
        best option: run nodes in multiple zones
      - Redis
        
        has many more features than Memcached: multi-AZ advanced queries, pubsub, persistence, backup, sorting
        
        supports encryption!
        
        no multi-threading support
        
        use-cases
        
        web-sessions store
        
        db caching
        
        leaderboards
        
        streaming data dashboards
        
        caching strategies
        
        write-through
        
        ensures that data is always fresh
        
        can fail with empty nodes
        
        can populate the cache with superfluous data
        
        lazy loading
        
        adds cache misses and decreases cache performance
        
        loads data into the cache only when necessary
        
        Adding TTL
        
        best of both worlds
        
        keeps data from getting too stale and requires that values in the cache are occasionally refreshed from the database
      - both support
        
        horizontal scaling
        
        caching
    - - is not for SQL usage!
      - usage implies heavy app changes!
      - cache invalidation is critical
      - for stateless architectures / to improve read RDS performance
- - - - hierarchical DB of users, groups, computers
      - security
        
        Kerberos, LDAP
        
        to attach Linux machines
        
        A bootstrap script that installs a Kerberos client package and performs a Realm Join
        
        DNS
      - HA
      - configuration overhead!
      - notes
        
        VMC Compute Gateway (CGW) Firewall Rules block traffic to all uplinks by default
        
        ADFS
        
        AD federation svcs
    - - AD w/o overhead + LDAP
      - multi-AZ
      - when
        
        For userbases more than 5,000
        
        want to establish a trust relationship with on-prem directories
      - notes
        
        compatible with AWS Single Sign-On / SAML
        
        provides a simple way to provide SSO for your users across AWS Organizations
        
        objects are organized in trees. Group of trees -> forest
        
        integrates with AWS analytics svcs
        
        to connect to on-prem
        
        MUST have a Direct Connect / VPN connection (site-to-site)
        
        setup trust
        
        one-way
        
        two-way forest
        
        AWS <-> on-prem
        
        replicated not supported
        
        when?
        
        part of U is stored on-prem, part on AWS
        
        HA
        
        achieved with replicating a Microsoft AD on EC2 and setting up a forest trust btw the actual AD and EC2
    - - basic AD features
      - large < 5k users
        small <500 users
      - low cost!
      - UC
        
        very simple AD
        
        LDAP compatibility
      - notes
        
        no MFA!
        
        no Trust relationships with on-prem
    - - UC
        
        single sign-on for on-prem employees
        
        adding EC2s to the domain
      - log-in from on-prem to AWS
        
        with less features than in Microsoft AD
      - must have existing on-prem AD
        
        +MUST have a Direct Connect / VPN connection (site-to-site)
  - - - UC
        
        org charts
        
        course catalogs
        
        device registries
        
        hierarchical data with complex relationships
      - fully managed
      - for developers
    - - sign-in apps
      - social media apps
      - managed user directory for SaaS
      - UC
        
        social media IAM connector
      - for mobile!
  - - - unique resource name
      - structure
        
        arn:partition:service:region:account_id
        
        '::' means the option is OMITTED
        
        when some res doesn't require this option (S3, etc)
        
        '/' in place of ':' are possible
        
        partition - aws OR ch (china)
        
        '*' means 'all'
    - - List of permissions that will then be attached to some Role
    - - defines the maximum permissions that the identity-based policies can grant to an entity
        
        only the permissions overlap between the PB and policy is effectively granted to U/R
        
        (same for SCP)
      - example
        
        AWS admin may access ALL res
        
        attached DynamoDB full A -> he can use only DynamoDB
      - notes
        
        unsafe! may be easily changed by other IAM users
        
        too complex if a lot of IAM principals
        
        only for U and R
        
        if U1 with PB creates U2, U2 has <= permissions than U1
  - - - choose one of the following identity sources
        
        3rd party identity provider
        
        active directory (AD)
        
        AWS SSO
    - - security assertion markup language
    - - centralized management
      - SAML
      - centralized audit with CloudTrail
      - relatively easy to setup (no STS involved)
    - - for mobile cases - Cognito
  - - - tmp creds are valid 15-12*60 mins
      - used to grant another U access to a R in your A or A that you generally own ('zone of trust')
      - to grant access outside of zone of trust
        
        fetch its AWS ID & External ID (secret key)
        
        define permissions
    - - AssumeRole API
        
        retrieve creds to impersonate a R you have access to
        
        steps
        
        create R -> grant access on this R to users -> U may assume this R
      - GetSessionToken
        
        for MFA
      - AssumeRoleWithWebIdentity
        
        creds for users logged with IdP
      - AssumeRoleWithSAML
        
        for SAML IdP
        
        SSO is a new replacement
        
        use-case
        
        user federation
- - - - ~Launch configuration
        
        BUT! has versioning built-in!
  - - - health checks + AS for EC2
        
        types
        
        maintain
        
        maintain current instance levels at all times
        
        3I -> 1 goes down -> 1 gets created
        
        scale manually
        
        use max / min number of instances
        
        dynamically
        
        based on real-time metrics
        
        on-schedule
        
        no scaling cool-down
        
        policies
        
        target metric tracking
        
        CPU >70%, etc
        
        simple
        
        slowly add one by one
        
        step
        
        customize scaling logic
        
        no cooldown! only warm-up - time between new instance comes up and next step is triggered
        
        notes
        
        scaling cooldown
        
        time btw scaling events
        
        healthcheck cooldown
        
        time at the start for instances to become available
        
        Default Termination Policy
        
        order of termination
        
        The instance launched from the oldest launch configuration
        
        health-check types
        
        EC2 status
        
        ELB (HTTP)
    - - AS for Dynamo, EMR, etc
        
        predictive scaling
      - policies
        
        scheduled
        
        step
        
        target tracking
    - - unified way to manage STACKS AS
  - - - switch helath check from ELB to EC2
    - - update launch template
        
        terminate EC2s manually (CloudFormation to automate)
        
        use EC2 instance refresh
        
        allows to set min % of running EC2 and update LT gradually
  - - - do smth before EC2 is running / is terminated
  - - - create new AS group for ELB
        
        allows to weight btw AS groups
      - add EC2 to the same AS group
      - create new ELB and use Route 53 CNAME weighted record
        
        allows to do full load testing
- - - - 1 acc pays bills for a group / individuals
        
        only 1 bill
        
        allows to do volume discounts
      - treat ALL A as a single A!
      - notes
        
        RIs and Saving Plan discounts aren't shared btw any A w/o sharing
        turned on
    - - includes CBilling
      - invited A must approve this
      - ability to apply SCP for member A to not leave the org
      - can't switch to CBilling feature
  - - - cannot be removed! only modified
        
        more secure than IAM policy that can be removed
      - applied to all users and roles in the account
    - - must be a single 'Statement'!
        
        if you have >1 SCPs, use JSON object array to put those
      - restrict root account activities of changing the root password or managing MFA settings if your accounts were created after 2017
      - allows to restrict access to certain features of sub-accounts
      - prerequisite: must enable all features in the organization
      - applied hirarchy-wise: if SCP is for Project unit, Team units that inherit PU will also have the SCP
      - Root user must have allow all SCP
      - may include allow / deny rules
      - doesn't affect
        
        management A
        
        service-linked R
      - aws:TagKeys
        
        allows to create Res only when they have specific tags
        
        used with ForAnyValue OR ForAllValues (all tags must be applied)
      - do not work to restrict any users or roles in the management account
    - - deny list
        
        all is ALLOWED
        
        can deny
        
        1 deny overrides ANY (implicit) allow
      - allow list
        
        all is DENIED
        
        can allow
        
        1 allow overrides ANY (implicit) deny
  - - - for future flexibility
    - - use tagging
      - enable CloudTrail on all A and send to the central S3 bucket
      - send CWatch logs to the central logging A
    - - to ensure consistent tags / audit resources, etc
      - & CloudWatch events to monitor non-compliant tags
    - - features
        
        immutable!
  - - - OrganizationAccountAccessRole must be MANUALLY created
    - - full access to this subordinate A
      - can be assumed by U in the Root OU
    - - Root OU
        
        OU dev
        
        member A
        
        OU prod
        
        OU finance
        
        OU hr
        
        management A
- - - - create a prioritized job queue
        
        create job definition, env vars, IAM, etc
        
        schedule the job
    - - scheduled / reoccuring HPC tasks w/o complex logic (no complex infra to setup, only EC2s)
    - - executor
        
        Fargate
        
        Spot & EC2 fleet in VPC
        
        make sure it can access ECS
        
        via VPC endpoint
        
        via NAT GW
        
        envs
        
        managed
        
        choose on-demand EC2 or Spot
        
        unmanaged
        
        fully customazible
      - serverless
      - any runtime packaged in Docker
      - EBS / instance store for disk
      - multi-node mode for HPC
        
        no Spot instances
    - - scheduling - CW events
      - orchestration - Step Functions
  - - - GB/s over a private net
    - - massive data transfer
    - - transfer btw on-prem and S3, EFS, FSx
        
        FSx - svc to launch 3rd party file-systems
        
        deployment options
        
        scratch
        
        temp
        
        high-burst
        
        cases: short-term usage
        
        persistent
        
        features
        
        replication in 1 AZ
        
        failover in minutes
        
        cases: long-term storage, sensitive data
  - - - instance store
        
        millions of IOps
      - EBS
        
        scales to 256k IOps with Block Express
    - - S3
      - EFS
        
        flexbile, scalable IOps
      - FSx for Lustre
        
        HPC optimized, millions of IOps
        
        backed by S3
        
        Lustre - FS for distributed computing
  - - - options
        
        ENA
- - - - incremental replication of on-prem S to AWS
      - min downtime when migrating on-prem VMs to AWS
      - creates AMIs from VMs
      - Replicated server volumes are encrypted in transit by TLS
    - - requires SMS connector to be loaded locally in vCenter (on a physical VM)
  - - - via network, etc
- - - - 1-7 days retention
      - data comes in shards
      - shard
        
        consumers
        
        5 transactions / sec for R OR 2Mb
        
        enhanced fan-out
        
        push-model
        
        2MB/s for R per enhanced consumer
        
        producers
        
        1000T / sec for writes OR 1Mb (including partition keys)
      - scales to Gbs of data / second
      - immutability
        
        once data is inserted, it CANNOT be deleted
      - storage 1-365 days
      - supports delayed delivery
    - - total stream capacity = sum of shards' capaicities
      - each record has
        
        seq number (auto-incremented)
        
        partition key
        
        a 256-bit hash
      - latency ~200ms
      - much cheaper than alternative with DynamoDB streams
      - real-time sink?
        
        use with Lambda
        
        is expensive
    - - non-AI related services
        
        yet, not a real-time ingestion svc (consider SQS)
    - - on-demand
        
        get #shards on-demand
      - provisioned
        
        manually provision #shards
  - - - RedShift, Athena, S3, 3rd party, OpenSearch, HTTP endpoint
      - Redshift sink means sending to S3 and COPYING it to Redshift
    - - NEAR real-time
        
        60s latency for non-full batches OR 1mb of data arrived
      - available analytic tools
      - data / failures may be written to S3
      - clientless. used solely as a BUFFER btw source and sink
    - - AWS, custom endpoint, Splunk, etc
    - - custom PUT
      - Kinesis Data Streams
  - - - near real-time
      - SQL / Flink (in Java) transforms
    - - streaming ETL
      - continuous metric generation
        
        ex: live leaderboard
      - responsive analytics
        
        query by simple criteria in data
  - - - producer
        
        any data generating device
        
        features
        
        can send
        
        video
        
        non-video
        
        audio feeds, images, or RADAR data
        
        1 P - N S
      - stream
        
        features
        
        real-time / batch / ad-hoc
        
        (optional) storage layer
        
        can carry
        
        audio, video, and similar time-encoded data streams
        
        1 S - N consumers
        
        sink
        
        EC2 for analysis
        
        KVS parser lib in custom solution
        
        Rekognition
        
        notes
      - consumer ('application')
        
        features
        
        can be EC2
        
        real-time processing OR persistent storage -> processing
        
        processes data such as fragments and frames
    - - no S3 sink!
        
        only with custom code
- - - - spot instances (primary) + on-demand (secondary) EC2s
        
        on-demand will turn on only when spot I are gone
  - - - warm standby
        
        pros
        
        reserved env is running in 'shadow' mode => secs/mins to recover
        
        env can be used as staging
        
        cons
        
        existing env may need to be scaled to match production load
        
        idea
        
        scaled-down version of prod env is always running
      - pilot light
        
        pros
        
        simple to implement
        
        configure Route 53 to direct traffic to another (identical) location
        
        cost-effective way to implement 'hot-site'
        
        idea
        
        keep on-hold a small copy of the running A (core functionality)
        
        cons
        
        mins/hours recovery
        
        sync between two infras
        
        manual intervention for recovery
      - multi-site
        
        pros
        
        ready all the time
        
        almost no intervention to recover
        
        cons
        
        expensive
        
        wasteful
        
        no down-scaling! two envs are identical all the time
      - backup & restore
        
        for small data sizes and generous RTO/RPO
    - - EBS
        
        RAID
        
        RAID6
        
        redundancy
        
        2 drives can fail
        
        IO
        
        fast R
        
        slowest W
        
        capacity
        
        (n-2)/n
        
        security bits!
        
        not recommended on AWS
        
        (n-1/)n
        
        RAID1
        
        redundancy
        
        1 drive can fail
        
        IO
        
        fast R
        
        fast W
        
        capacity
        
        50%
        
        2 drives
        
        requires 2x the required volume space
        
        (n-1/)n
        
        RAID5
        
        redundancy
        
        1 drive can fail
        
        IO
        
        fast R
        
        slow W
        
        capacity
        
        (n-1/)n
        
        security bits!
        
        not recommended on AWS
        
        3 drives
        
        RAID0
        
        redundancy
        
        None can fail
        
        IO
        
        fastest R
        
        fastest W
        
        capacity
        
        100%
    - - recommendations
        
        Route 53 / ELB health-checks
        
        up-to-date AMIs
        
        ELBs
        
        reserved instances
        
        the only way to guarantee resource availability when disaster occurs
        
        design for horizontal scaling
    - - order of deceasing fault tolerance
        
        DynamoDB
        
        Aurora
        
        multi-AZ RDS
        
        redundancy
        
        automatic recovery
        
        inherent HA
  - - - 100 000 RPS (req per second)
      - compute
        
        ALB
        
        ASG / ECS
        
        slow, bootstrap
        
        Fargate
        
        faster ECS
        
        Lambda
        
        1000 concurrent
        
        10 000 RPS
        
        DB
        
        Dynamo
        
        autoscaling
        
        on-demand
        
        other
        
        provisioned
        
        persistence
        
        msg buffers
        
        SQS, SNS
        
        unlimited
        
        SQS FIFO
        
        3000 RPS (with batching)
        
        Kinesis
        
        1/2 MB/s for in/out
    - - S3
        
        KMS limits if encr
        
        3500 PUT, 5500 GET per prefix / s
- - - - 256Kb of text, any format
        
        2Gb if Java SDK!
      - fail-safe queue
      - pull-based
      - msgs are kept 1 min - 14 days
        
        default - 4 days
      - visibility timeout
        
        a M becomes invisible after reader 1 reads it and gets deleted if the read transaction completes within the timeout range
        IF NOT - msg becomes visible for reader 2 to read
        
        leads to duplicate reads!
        
        0 sec - 12 hours
        
        default - 30s
      - polling
        
        long
        
        API hangs till a M arrives at the Q
        
        short (standard)
        
        API returns immediately
        
        each time EC2 polls SQS => charge!
      - not interactive! communication is done btw servers (such as EC2)
      - msg-oriented API
    - - standard
        
        = 1 delivery
        
        inf. transactions per second!
        
        order not guaranteed!
      - FIFO
        
        =1 delivery
        
        auto-deduplication
        
        msg persists until deleted
        
        guaranteed order
        
        limited to 300 T / second
        
        support msg groups (multiple ordered msg groups in a queue)
        
        12 hours - MAX visibility timeout
        
        8 transaction per operation <-> 2,400 transactions/sec
        
        max 10 transactions per operation in batch mode
    - - eventually consistent. use DelaySeconds to defer new msg visibility for consumers
        
        use visibility timeout to delay msg returned by a consumer to SQS
      - integration with AutoScaling
        
        backlog per instance
        
        to control instances based on SQS backlog
      - integration with Lambda
        
        Event source mapping for Lambda = long polling for batch of values
        
        set visibility timeout to 6x Lambda timeout
      - to use a DLQ
        
        set up another SQS
        
        with Lambda
        
        not DLQ! only async invocations
        
        use Destinations
  - - - for new apps - use Step Function
    - - 1 year retention
      - task-oriented API
      - task is NEVER duplicated
      - similar workflows can be grouped in one 'domain'
      - no programming lang restrictions (HTTP API)
      - fan-out delivery
    - - workflow starters
        
        initiate a W
        
        examples
        
        place an order, search for something
      - deciders
        
        control the flow of activity tasks in a workflow
      - activity workers
        
        carry out activity
  - - - manage notifications
      - integrates with SQS, HTTP endpoints
      - has 'topic' with multiple deliverables
      - redundancy (3 AZs)
      - push-based delivery
      - inexpensive (pay-as-you-go)
      - fan-out msg to all services interested
      - supports fan-out pattern
        
        solutions
        
        SNS + SQS
        
        ordering + deduplication
        
        SNS FIFO + SQS FIFO
        
        fan-out + ordering + deduplication
    - - POSTs 2 kinds of headers
        
        notification
        
        confirmation
      - requires parsing custom HTTP headers
      - downstream must be idempotent to handle SNS retries
      - requires the HTTPS subscriber to present a trusted CA-signed certificate
  - - - supports MQTT (IoT), JMS, WebSocket protocols
      - designed as a replacement for on-prem MB
      - HA within a region
- - - - ~EKS
        
        the difference in IAM management: for ECS it is provided by default
      - AWS only!
      - handles Dockerized apps
      - leverages AWS services (EKS does this internally!!)
      - considered easier than EKS
    - - cluster
        
        collection of ECS resources / Fargate instances
      - task
        
        1 working copy of some C
      - task definition
        
        Dockerfile for ECS, can define multiple C
      - service
        
        ~deployment in k8s
        
        networking
        
        none
        
        bridge
        
        docker net
        
        awsvpc
        
        every task gets its own ENI and a private IP
        
        notes
        
        default for Fargate!
        
        has simplified networking, SGs, VPC flow logs, etc
        
        host
        
        use underlying EC2 interface
        
        scaling
        
        types
        
        target tracking
        
        based on CW metric
        
        step
        
        based on CW alarm
        
        scheduled
        
        works on TASK level, not instance (unlike EC2 AS)
      - registry
        
        storage for imgs
      - IAM role
        
        task role
        
        unique permissions for a task
        
        instance profile
        
        used by EC2
        
        scenario
        
        common for all instances: making API calls, sending logs, etc
    - - can apply policy / task (granular, instance-wise)
      - to specify secretOptions in task def
        
        ParameterStore
        
        Secrets manager
    - - doesn't support third-party add-ons
      - Task role > EC2 instance role while granting ECS cluster permission to use resources during C execution
      - provides tagging at the repository level, not at the individual container image level
      - ALB integration
        
        Dynamic port mapping
        
        allows to run N tasks on 1 EC2 on different ports
        
        when
        
        rolling updates / maximize utilization / etc
  - - - automatically provisions ECS / EKS resources
    - - runs on ECS, EKS
      - isolation and security
      - no costs when idle
      - uses EFS for persistent storage, ephemeral storage for non-persistent
    - - no GPUs!
      - constraints due to serverless nature
    - - EC2 Launch Type
        
        manual management of resources
  - - - works with on-prem deployments
      - HA
      - integrated with IAM
      - pay for storage + data transfer
- - - - they are assigned on user account itself
    - - not resources!
    - - but!
        
        create shared IAM role and add another account to its trust policy
    - - Assuming a role means GIVING UP previous R and using only this shared one!
      - RBP simply allows to do smth to a resource
    - - RBAC
        
        role-based access control
        
        attach roles to U/G
      - ABAC
        
        attribute-based access control
        
        allow actions based on tags that U / target objects have
  - - - def
        
        an individual
      - cannot have Trust Policy
    - - a set of U
      - a way to attach policies to multiple users
      - notes
        
        A group can contain many users, and a user can belong to multiple groups
        
        can contain only users, not other groups
        
        There's a limit to the number of groups
    - - Permission to a specific R
      - best practice: use roles to delegate permissions
      - notes
        
        may be shared
    - - def
        
        a document with JSON array of permissions on R for U/G/R
      - types
        
        AWS managed
        
        Custom managed
        
        Inline
        
        for 1 U/R
        
        possible to share
        
        resource-based
        
        separate policy for an S3 bucket, SQS, etc
        
        such as Trust Policy
        
        identity based
        
        such as permissions policy
      - structure
        
        JSON with 'Version' field and 'Statement' obj array
        
        Statement
        
        Effect
        
        Action
        
        NotAction
        
        scenario: deny all in NotActions and allow SOME in Action (these coexist)
        
        Resource
        
        adding ${aws:username} selects ALL resources with current user in their name
        
        Condition
        
        Policy variables
      - notes
        
        explicit DENY > allow
        
        Access Advisor - optimize policies permissions
        
        Access analyzer
        
        explore shared with entities outside zone of trust
  - - - Trust policy of the R itself must allow the U to assume the R
        
        is met if User's arn is specified in the Principal element of the Trust Policy
        
        Trust Policy says WHO can access the ROLE
        
        example of a resource-based policy
- - - - status check
      - features
        
        metrics
        
        available
        
        disk
        
        CPU
        
        net IO
        
        std monitoring: each 5m, detailed - 1m
        
        custom
        
        trigger: 1s - 1m (by def)
        
        ex: RAM
        
        use unified CW agent for extra system metrics
        
        has centralized config in Parameter Store
        
        cannot sent to Kinesis! use Kinesis agent instead
    - - alarms
        
        notes
        
        can't monitor the state of resources
        
        features
        
        trigger actions: EC2 / Autoscaling / SNS
        
        can be intercepted with CW Events
      - events
        
        notes
        
        can monitor the state of resources
        
        can be scheduled
        
        sinks
        
        Lambda
        
        Step F
        
        Kinesis
        
        features
        
        intercept API calls from CT
        
        monitor state change of AWS resources
    - - defs
        
        log group
        
        assembly for agg app logs
        
        log stream
        
        set of log units (log files, containers, etc)
      - features
        
        expiration policy
        
        KMS encr
        
        sinks
        
        S3 / Kinesis / Lambda
        
        S3
        
        buckets MUST be encr with AES-256 (SSE-S3), not SSE-
        KMS!
        
        logs available in 12 hours!
        
        CreateExportTask - API call to export logs from S3
        
        for real-time - Logs Subscriptions & Lambda
        
        makes multi-account & multi-region logs available!
        
        CW + Logs subscr + Kinesis
        
        metric filters
        
        query logs and create CW dashboards
        
        can be from on-prem
  - - - enabled by default
      - events are stored 90 days
      - S3 usage
        
        SSE-S3 is applied by default when logging to S3
        
        SSE-S3 managed by AWS
        
        cross-account logging
        
        bucket policy is mandatory
        
        to give one A access
        
        options
        
        create cross-A role and assume it
        
        edit the bucket P
        
        delivery in 5 mins
      - can be streamed to CW logs
        
        and analyzed for unusual activity (+alarms, etc)
      - Organization trail
        
        centralized CT in the management A that monitors all members
      - <15 mins to deliver events
        
        use CW events to mitigate this
    - - insight
        
        analyze WRITE events to detect UNUSUAL behavior
      - data
        
        GET/PUT/Invoke (for Lambda), etc
        
        not logged by default!
      - management
        
        ops on resources: delete, create, etc
        
        logged by default!
        
        separated into 2 groups
        
        read
        
        other
        
        write
        
        modification of R
  - - - monitor what happens in resources
- - - - VPC
        
        single-AZ
        
        storage
        
        EMRFS - S3 (native)
        
        permanent
        
        server-side encr
        
        HDFS - EBS
        
        temporary
      - Hive - DynamoDB
  - - - master
        
        logs must be archived!
        
        otherwise - if master goes down - the whole C goes down
        
        must be configured during setup
      - core
        
        store data
        
        run tasks
      - task
        
        (optional)
        
        runs tasks
      - notes
        
        all nodes communicate with each other
    - - taxonomy
        
        1
        
        long-running
        
        transient
        
        just for 1 job
        
        2
        
        single big
        
        many small C
        
        for several concurrent jobs
      - instance config
        
        uniform groups
        
        auto-scaling!
        
        select I types and purchasing for each node
        
        instance fleet
        
        NO auto-scaling
        
        select target capacity, I types, purchasing
- - - - identity
        
        actual granting of access to AWS services
      - user
        
        about actual users sign-up/sign-in : email, ID, etc
    - - recommended for use by ALL mobile devices
      - JWT - javascript web token
      - uses SNS to sync user data btw devices
      - allows to bypass creation of IAM U
      - replaces a TVM - Token Vending Machine
      - can associate U with IAM R / P
    - - auth via FB -> receive response from user pool with JWT -> send JWT to identity pool -> receive temporary credentials -> access AWS
  - - - Active directory, etc
- - - - data discovery with Athena / EMR / Redshift Spectrum
      - actual ETL with Glue
- - - - fully-scripted IaaS (Terraform-like definition)
      - sample AWS architectures are hosted AI-hub-like website
      - alternative for simple cases
        
        Elastic Beanstalk
        
        deploy a basic apps / websites w/o thinking about infrastructure
        
        orchestration S to easy deploy scalable web apps
        
        features
        
        multiple envs (dev/prod) within one app
        
        fully customizable
        
        free to use, pay for provisioned resources
        
        support for many platforms
        
        programming: Go, Java, Java with Tomcat, etc
        
        Docker
        
        can write custom platform
        
        great to 'replatform' on-prem app
        
        architecture models
        
        single I (for dev)
        
        LB+ASG
        
        for web-apps
        
        ASG
        
        for non-web apps
        
        ex: scale based on SQS load
        
        combine these to decouple architecture!
        
        deployment
        
        Blue/green
        
        Zero downtime
        
        Beanstalk can swap URLs (DNS swap) in R53 to switch env
        
        rolling
        
        immutable
        
        with new instances
        
        in-place all-at-once
        
        notes
        
        the only release way with downtime is 'all at once' - when a new version is deployed to all instances at once
        
        cannot deploy Lambda
      - defs
        
        template
        
        JSON/YAML with full env definition
        
        stack
        
        the entire env that is managed as a single unit
        
        types
        
        nested stack
        
        used when stack outputs should be limited within the stack group
        
        scenario
        
        component reuse
        
        2 more items...
        
        cross-stack
        
        when stacks have different lifecycles
        
        scenarios
        
        when you need to export values to many stacks
        
        change set
        
        a dry-run summary of changes
        
        stack policies
        
        JSON with permissions on resources in a stack
        
        note!
        
        can be created with stack via CLI or console
        
        can be added to a stack ONLY via CLI
        
        cannot be removed! only modified
        
        drift
        
        deviation of stack's resources from the template
      - best practices
        
        modify the stack itself rather than created resources
        
        attach stack policies
        
        use AWS Python helpers to examine stack
        
        make use of change sets
        
        track changes to stack in VCS
        
        use StackSet to integrate 3rd party (monitoring) tools into multiple accounts
      - notes
        
        To integrate external services we can use a custom resource
        
        to preserve EBS on EC2s if stack is being deleted
        
        Add a DeletePolicy attribute in the template and specify "Snapshot"
        
        to implement conditionals
        
        use Fn::If
        
        no switch/case available
        
        to add external svc with a REST API, add a custom resource in your T
        
        such as Lambda
        
        minimal downtime
        
        with Chef
        
        install Chef client on EC2s and run updates via Chef
        
        manages ASG, not EC2s!
        
        CreationPolicy
        
        success conditions for launching new EC2s
        
        UpdatePolicy
        
        update strategies for EC2s
        
        to update EC2 in ASG - create launch config & use UpdatePolicy
        
        DeletePolicy
        
        what happens when CF template is deleted
        
        options
        
        Retain
        
        keep a resource
        
        Snapshot
        
        delete, but make snapshot
        
        Delete
        
        default for all res, except RDS
        
        to delete S3 bucket, empty it first
      - features
        
        stack drift detection
        
        rollback on cloudWatch alarms
        
        tracking changes in stacks via AWS Config
        
        deployment
        
        IAM
        
        (default) acts as IAM principal who deployed
        
        separate IAM role for the stack
        
        to create IAM res in stack
        
        add capabilities!
        
        CAPABILITY_IAM
        
        CAPABILITY_NAMED_IAM
        
        custom resources (Lambda)
        
        to address custom cases
        
        fetch an AMI id, empty S3 bucket, access on-prem, etc
        
        CloudFormer
        
        create T from AWS resources
        
        ChangeSets
        
        preview changes before they get applied
        
        StackSets
        
        deploy stacks cross-A cross-R
        
        Stack policies
        
        prevent accidental updates / deletes
        
        integration with secrets manager
        
        gen secret
        
        reference S in your resource
        
        link the res and the S
    - - (cheap way)
        
        2 AZ, 1 BH, 1 I + autoscaling
        
        not 100% FT
      - (more expensive way)
        
        2 AZ, NLB, 2 BH, 1 I + failover
    - - Serverless Application Model
      - alternative to Cloud Formation for building serverless architectures
      - features
        
        supports everything Cloud Formation does
        
        allows to test locally
        
        run Lambda, API GW, Dynamo locally
        
        pkg and deploy using CodeDeploy
        
        YAML definition
      - for multi-cloud
        
        use 'serverless framework'
      - for integration btw AWS and 3rd party services
        
        EventBridge
        
        E: Datadog integration
      - solutions
        
        Code pipeline
        
        CodeCommit + Code Build + CloudFormation & SAM
        
        CodeDeploy -> Lambda v1, v2
        
        Dynamo
        
        API GW
  - - - serverless, scalabie
      - HA
      - app -> state machine
    - - does not integrate with Mechanical Turk
      - vs SWF
        
        SWF
        
        Mechanical Turk
        
        externals signals need intervene in workflow
        
        child process returns values to parent
    - - API GW
      - CW E
      - SDK / CLI
- - - - Fan-out
        
        app should send messages to a msg buffer rather than directly to multiple processing locations
        
        example
        
        app sends to SNS, SNS broadcasts to all SQS subscribers
        
        app sends to SQS
        
        SQS sends to Lambda
        
        Lambda calls all required downstream svcs
      - DLQ
        
        examples
        
        SQS sends msgs that exceed maxReceiveCount to DLQ (dead-letter queue, which is another SQS queue)
        
        Lambda sends results of failed async invocations to SNS / SQS
        
        SNS publish fails -> msg is sent to SQS for further investigation
        
        a place where undelivered things arrrive
        
        some svc (for example, lambda) watches this DLQ and decides what to do next
    - - when something happens, notify those concerned
      - example
        
        S3 notifications
        
        consumers
        
        SNS / SQS / Lambda
        
        event types
        
        new object / deletion / update / restore / replicate