Please enable JavaScript.

Coggle requires JavaScript to display documents.

AWS - Coggle Diagram

- - - - Run on-demand
      - Limited runtime
      - Auto scaling
      - Multi-language support
        
        Node.js, Python, Java, C#, Golang, Ruby
      - Environmental variables
        
        Up to 4KB
    - - Synchronous
        
        CLI, SDK, API Gateway, Application Load Balance, S3 Batch, CloudFront
      - Asynchronous
        
        • S3, SNS, CloudWatch Events / EventBridge
        
        Idenpotent response incase of retries
        
        Retries show up on CloudWatch logs
      - Event Source Mapping
        
        Link that lets some AWS services invoke Lambdas
        
        Configure
        
        • Discard old events
        
        • Restrict the number of retries
        
        • Split the batch on error to work around lambda timeout issues
        
        Kinesis
        
        Creates an iterator for each shard
        
        SQS
        
        Long-polling
    - - Failed or successful events
      - • Send to: SQS, SNS, Lambda, EventBridge
    - - IAM
        
        Attach to lambda so it can access other services
      - Resource Policy
        
        Enable other services to access lambda
    - - Displayed in CloudWatch Metrics
      - Invocations, Durations, Concurrent Executions
      - Error count, success rates, Throttles
      - Async delivery failures
      - Iterator Age (Kinesis and DynamoDB Streams)
    - - Enable Active Tracing in lambda config and will run X-Ray daemon for you
      - IAM Role
        
        ASWXRayDaemonWriteAccess
      - Env Vars
        
        _X_AMZN_TRACE_IC: contains the tracing header
        
        AWS_XRAY_CONTENT_MISSING: default, LOG_ERROR
        
        AWS_XRAY_DAEMON_ADDRESS: the X-Ray Daemon IP_ADDRESS:PORT
    - - Node.js or Python
      - Max execution time 5 - 10s
      - Use Case
        
        • Customize the CDN content
        
        • Website security and privacy
        
        • Search Engine Optimisation (SEO)
        
        • Intelligently route across origins and data centres
        
        • Bot mitigation
        
        • Real-time image transformation
        
        • A/B Testing
        
        • User Authentication and Authorisation
        
        • User Prioritisation
        
        • User Tracking and Analytics
      - Upto 50Mb
      - Max Memory 10GB
      - Max package size 10GB
      - Access to request body
      - File system access
      - 1000s requests / s
    - - Lambda has its own AWS VPC
      - Can't access resources in your VPC
        
        Use Elastic Network Interface (ENI)
        
        AWSLambdaVPCAccessExecutionRole
      - Lambda in your VPC
        
        NAT
        
        IGW
        
        Access Internet
        
        Deploying lambda in a public subnet does NOT give it internet access or a public IP - need NAT
    - - RAM
        
        128MB to 10GB in 1MB incr
        
        The more RAM the more CPUs you get
        
        1792MB <-> one vCPU
        
        If you application is CPU bound increase the RAM
        
        Timeout 3 to 900 seconds
      - Execution Context
        
        Maintained after lambda finished
        
        /tmp dir 10GB max
        
        S3 for permanent persistence
        
        UC db connections, https clients
    - - Create layers for larger libraries
      - Custom runtimes c++ rust
      - Layers reused across lambdas
    - - Upto 1000 concurrent executions
      - Synchronous Throttle error (429) if you go over
      - Asynchronous throttle error -> DLQ
      - Can request higher limit
      - Cold start
        
        1st request higher latency
      - Provisioned Concurrency
        
        No cold start
    - - Inline
        
        Code.ZipFile
      - Zip
        
        Refer to S3 location
    - - Work on version is $LATEST
        
        Mutable
      - Publish a version
        
        Immutable
      - Each version has own ARN
      - Aliases
        
        Pointers to lambda versions
        
        Mutable
        
        Enable canary deployment
        
        Own ARNs
        
        Can't reference other aliases
        
        Code Deploy
        
        Deployment Types
        
        Linear
        
        Grow traffic until 100%
        
        Canary
        
        X% and then 100%
        
        AllAtOnce
        
        immediate
    - - Unique endpoint
      - Public internet
      - Supports Resource based policies
      - Supports Cors
      - Throttle using reserved concurrency
      - Auth Type
        
        NONE
        
        All public and unauth access
        
        Resource based policy alway in effect
        
        AWS_IAM
        
        Authenticate and authorise
        
        Principle's identify-based and resource-based policies are evaluated
        
        Principle must have lambda:InvokeFunctionUrl permissions
    - - An Amazon API Gateway Lambda authoriser (formerly known as a custom authoriser) is a Lambda function that you provide to control access to your API. A Lambda authoriser uses bearer token authentication strategies, such as OAuth or SAML. Before creating an API Gateway Lambda authoriser, you must first create the AWS Lambda function that implements the logic to authorise and, if necessary, to authenticate the caller
  - - - • Supports Websocket
      - • Versioning APIs v1, v2
      - • Handle different environments
      - • Handle security (Authentication and Authorisation)
      - • Create API keys and request throttling
      - • Swagger / Open API import to quickly define APIs
      - • Transform and validate requests and responses
      - • Generate SDK and API specifications
      - • Cache API responses
      - HTTP API cheaper than REST Api
      - Can create single interface for all micro-services
    - - Lambda
      - HTTP
        
        Expose endpoints in backend
      - AWS Service
        
        Step Function
        
        SQS
        
        Kinesis
        
        S3
    - - Edge Optimised
        
        Global clients
        
        Requests routed through CloudFront
        
        Gateway still lives in one region
      - Regional
        
        Clients in same region
      - Private
        
        Accessed only from VPC using interface ENI
        
        Used resource based policy for access
    - - User Authenticated
        
        o IAM Roles (internal applications)
        
        o Cognito (identify for external users – example mobile users)
        
        o Custom Authoriser (your own logic)
      - AWS Certificate Manager (ACM)
        
        Edge optimised the certificate must be in us-east-1
        
        Regional endpoint certificate in API Gateway region
        
        Setup CNAME or A-Alias in Route 53
        
        Only use IAM if in a region not supported by ACM
    - - Need to deploy to effect changes
      - Deploy to many stages
      - Stages can be rolled back as a history of deployments is kept
      - Stage variables
        
        Once TEST
        is passed change the stage variable value to PROD
        
        Config for API Gateway
        
        Used in
        
        Lambda ARN
        
        HTTP Endpoint
        
        Parameter Mapping Templates
        
        Format the data response
        
        Use cases
        
        Configure endpoint stages
        
        Pass config params to lambda through mapping templates
        
        ${stageVariables.variableName}
      - Canary Deployments
        
        Specify which stage gets %
    - - MOCK
        
        API Gateway returns a response without sending the request to the backend. Simulates the use of components for integration testing
      - HTTP / AWS
        
        Configure both the integration request and integration response
        
        Setup data mapping using mapping templates for request and response
        
        Mapping Templates
        
        • Used to modify request / responses
        
        • Rename and modify query string params
        
        • Modify body content
        
        • Add headers
        
        • Uses Velocity Template Language (VTL): for loop, if else etc
        
        • Filter output results (remove unnecessary data)
        
        Exposes actual endpoints
      - AWS_PROXY
        
        o Incoming request from client in the input to Lambda
        
        o The function is responsible for the logic of the request / response
        
        o No mapping template, headers, query string params are passed as arguments
      - HTTP_PROXY
        
        o No mapping template
        
        o Http request is passed to the backend
        
        o Http response from the backend is forwarded by API Gateway
        
        o Possibility to add HTTP headers if need be (e.g. API Key)
    - - Define API Spec - json or yaml
      - Use for request validation
    - - Improve API performance
      - Reduce calls to backend
      - Default TTL 300s
      - Override cache settings per method
      - Encryption option
      - 0.5GB to 237GB
      - Expensive use for prod only
      - Invalidate
        
        Cache-Control:max-age=0
        
        Policy otherwise anyone could invalidate it
    - - Usage Plan
        
        o Who can access one or more deployed API stages and methods
        
        o How much and how fast they can access them
        
        o Uses API key to identify API clients and meter access
        
        o Configure throttling limits and quota limits enforced on individual clients
        
        Configuration
        
        1 Create or import APIs to distribute to customers
        
        2 Create usage plan with throttle and quote limits
        
        3 Associate API stages and API keys with usage plan
      - Alphanumeric string distributed to customers
    - - Cloudwatch Logs
        
        Request and response info
        
        Enable at stage level (ERROR, INFO)
        
        Override on API basis
      - X-Ray
        
        Extra info about requests
      - CloudWatch Metrics
        
        Detailed metrics
        
        Cache hit and miss count
        
        Integration latency
        
        Request and response time to backend
        
        Latency - request and response time to client
        
        4xx and 5xx errors
        
        Client Errors
        
        o 400 – Bad Request
        
        o 403 – Access Denied
        
        o 429 – Quota exceeded
        
        Server Errors
        
        o 502 – Bad Gateway exception – incompatible output returned from lambda proxy
        
        o 503 – Service unavailable exception
        
        When no targets are registered
        
        o 504 – Integration Failure – e.g. Endpoint Request Timedout exception (29secs timeout)
    - - Enables to receive API calls from another domain
      - OPTIONS header
        
        o Access-Control-Allow-Methods
        
        o Access-Control-Allow-Headers
        
        o Access-Control-Allow-Origin
      - Enabled through console
      - Restrict access on methods
    - - Authentication and Authorisation
        
        • IAM:
        
        o Great for users / roles already within your AWS account + resource based policy for cross account
        
        o Handle authentication and authorization
        
        o Leverages Signature v4
        
        • Custom Authoriser
        
        o Great for 3rd party tokens
        
        o Flexible in terms of what IAM policy is returned
        
        o Handle Authentication and Authorisation in the lambda function
        
        o Pay per lambda invocation, results are cached.
        
        • Cognito User Pool
        
        o You manage your own user pool (backed by Facebook, Google login etc)
        
        o No need to write any custom code
        
        o Must implement authorization in the backend
  - - - • Users given an identity to interact with our web or mobile application
      - Cognito vs IAM
        
        100s of users
        
        Mobile users
        
        Authenticate with SAML
    - - Summary
        
        • Creates a serverless database of users for your app
        
        • Username and password
        
        • Password reset
        
        • Email and mobile verification
        
        • MFA
        
        • Federated Identities – Facebook, Google, Amazon
        
        • Block users if credentials are compromised elsewhere
        
        • Login sends back a JWT
        
        • Uses a hosted UI for login
        
        • Can use pre and post lambda triggers with CUP
        
        Authentication
      - Integrates
        
        API Gateway
        
        Application Loadbalancer
        
        Use Https listener
        
        authenticate-oidc
        
        authenticate-cognito
      - Lambda Triggers
        
        Events
        
        Pre-auth
        
        Validation
        
        Post-Auth
        
        Logging
        
        Pre Sign-up
        
        Post-Confirmation
        
        Welcome messages
        
        Migrate user
        
        Message Trigger
        
        Pre Token Trigger
      - Hosted Domain
        
        Certificate in ACM in us-east-1
        
        Add to app integration section
      - Adaptive Authentication
        
        Sign-ins are given a risk score
        
        Logging in CloudWatch logs
      - Issues JWT Tokens
    - - Give users temp credentials
      - Pool contains
        
        Facebook, Google etc
        
        Custom auth server
        
        Guest access
      - Users can access AWS Services directly or via API Gateway
        
        IAM creds applied to users
      - Authorisation
      - Default IAM roles for users
    - - Amazon Cognito Sync is an AWS service and client library that enables cross-device syncing of application-related user data. You can use it to synchronize user profile data across mobile devices and the web without requiring your own backend. The client libraries cache data locally so your app can read and write data regardless of device connectivity status. When the device is online, you can synchronize data, and if you set up push sync, notify other devices immediately that an update is available.
  - - - Type = 'TASK'
    - - • Choice State – Test for a condition to send to a branch
      - • Fail or Succeed State – Stop execution with failure or success
      - • Pass State – Simply pass its input to its output or inject some fixed data, without performing work
      - • Wait State – Provide a delay for a certain amount of time or until a specified time/date.
      - • Map State – Dynamically iterate steps
      - • Parallel State – Begin parallel branches of execution
    - - o Definition issues – no matching rule
      - o Task failures – exception in a lambda function
      - o Transient issues – network partition events
      - State can report its own errors
      - Error codes
        
        o States.ALL – matches any error name
        
        o States.Timout.Task – ran longer than timeout
        
        o States.TaskFailed – execution failure
        
        o States.Permissions – insufficient privileges to execute code
      - Retry
        
        • Evaluated from top to bottom
        
        • ErrorEquals – match the error
        
        • IntervalSeconds – delay before retrying
        
        • BackoffRate – multiple the delay after each retry
        
        • MaxAttempts – default to 3 set to 0 for never retried
        
        • When max attempts are reached the Catch kicks in
      - Catch
        
        • Evaluated from top to bottom
        
        • ErrorEquals – match the specific error
        
        • Next – state to send to
        
        • ResultPath – A path that determines what input is sent to the state specified in the Next field
    - - Append .waitForTaskToken to the resource field
  - - - o Data storage
        
        AppSync and DynamoDB
      - o Authentication
        
        Cognito
      - o Machine learning
  - - - • AssumeRole: Assume roles within your account or cross-account
      - • AssumeRoleWithSAML – return credentials for users logged in with SAML
      - • AssumeRoleWithWebIdentity
      - o Return creds for users logged in with Facebook, Google etc
      - o AWS recommends against this, and using Cognito Identity Pools instead
      - • GetSessionToken – for MFA from a user or AWS root user
      - • GetFederationToken – get temp creds for a federated user
      - • GetCallerIdentity – get details about IAM user or role used in the API call
      - • DecodeAuthorizationMessage – decode error message when an AWS API is denied
  - - - S3
      - SNS
      - Lambda
      - IAM
  - - - o Business intelligence / analytics / reporting
- - - - Data centers
  - - - Public
      - Private
    - - Connect two VPC with non overlapping IP ranges, non transitive
    - - Provide private access to AWS Services within VPC
      - S3
      - DynamoDB
    - - Ip traffic informtation
  - - - Resource Policies are only
        trust polices
    - - ${aws:username} - Policy Variable
  - - - EC2
      - ENI
        
        Elastic Network Interface
  - - - Over public internet between on-prem and AWS
  - - - Summary
        
        • Manages the keys for us
        
        • Fully integrated with IAM for authorization
        
        • Control access to your data
        
        • Able to audit KMS key usage using CloudTrail
        
        KMS stores the CMK, and receives data from the clients, which it encrypts and sends back
      - Key Types
        
        Symmetric
        
        Asymmetric
        
        RSA and ECC Key pairs
      - Automatic Key Rotation
      - Keys not shared across regions
      - Performs encryption and decryption
        
        4KB data limit
        
        4KB use Envelope encryption
      - Use S3 Bucket Key to save API calls to KMS
    - - Summary
        
        • Secure storage for config and secrets
        
        • Optional seamless encryption using KMS
        
        • Serverless, scalable, durable easy SDK
        
        • Version tracking of config / secrets
        
        • Security through IAM
        
        • Notifications with EventBridge
        
        • Integration with CloudFormation
        
        • Can store as a hierarchy
      - Number of parameters
        
        Standard 10_000 and 4KB max
        
        Advanced 100_000 and 8KB max
      - Can assign TTL
      - Can pull secrets from Secret Manager
    - - Summary
        
        • Newer service meant for storing secrets
        
        • Can rotate secrets every X days
        
        • Auto generation of secrets on rotation – uses lambda
        
        • Integration with RDS
        
        • Secrets are encrypted using KMS
        
        • Mostly meant for RDS integration
        
        • Can multiple across AWS Regions
        
        • Secret manager keeps read replicas (Multi-AZ) in sync with the primary secret
        
        • Ability to promote a read replica Secret to a standalone secret
- - - - Managed Service
      - Backups
        
        Automated
        
        Retention 1 to 35days
        
        Enabled by default
        
        Backup data stored in S3
        
        Manual
        
        Store database snapshots manually
      - Multi-AZ
      - Scaling Vert and Horz
      - Storage backed by EBS
      - Auto Failover to standby
      - OS updates to standby and then swap
      - Read Replicas
        
        Mutli-AZ Disaster Recovery
        
        Promote to own DB
        
        Cross Region (Pay more)
        
        Cross AZ
        
        Witin AZ
        
        Increase Performance
        
        Asynchonous replication
        
        Encryption at rest - KMS
      - Can turn on auto scaling
      - Auto OS Updates
    - - Data Warehouse
      - Column compression
      - Nodes
        
        Single
        
        Multi
      - Backup (S3) 35 days
      - Single AZ
      - UC - Business Intelligence
      - 3 copies of data
    - - Compatible
        
        MySQL 5 x better
        
        PostgreSQL 3 x better
      - Up to 15 replicas
      - 10GB to 128TB
      - 2 copies of data in each AZ (min 3 AZs)
      - Self-healing
      - 1 master lots readers
      - Serverless
      - No SSH
      - Share snapshots across accounts
      - UC - simple, cost-effective, infrequent / unpredictable workloads
  - - - Basics
        
        SSD Storage
        
        Spread over 3 data centres - geo separated
        
        Serverless
        
        Max item size - 400KB
        
        Fully Managed
      - Structure
        
        Tables
        
        Indexes
        
        Local Secondary Index (LSI)
        
        Alternative sort key for table (same as partition key)
        
        Created at the same time as the table
        
        Can have up to 5 LSIs per table
        
        Uses WCUs and RCUs of the main table
        
        No special throttling considerations
        
        Global Secondary Index (GSI)
        
        Alternative primary key from the base table
        
        Can be created after table creation
        
        Must provision WCUs and RCUs for the index
        
        If writes are throttled on GSI then main table is throttled even if WCU on main table is fine
        
        Data types
        
        Supported
        
        • Scalar Types: String, Number, Binary, Boolean, Null
        
        • Document Types – List, Map
        
        • Set Types – String Set, Number Set, Binary Set
        
        Keys
        
        Primary
        
        Unique for each item
        
        Diverse so data is distributed
        
        Partition Key + Sort Key
        
        Combination unique for each item
        
        E.g. User ID + Game ID
      - Capacity Modes
        
        Provisioned
        
        o Need to plan capacity before hand
        
        o Pay for provisioned read and write capacity units
        
        Can use 'Burst Capacity' for temp high load
        
        If exceeded get - ProvisionedThroughputExceededException
        
        Try exponential backoff
        
        o Specify number of read/writes per sec
        
        On demand
        
        o Read/Writes automatically scale up/down with your workloads
        
        o No capacity planning needed
        
        o Pay for what you use, more expensive 2.5x
        
        Can swap every 24hrs
        
        Writes
        
        Write Capacity Units
        
        WCU = one write per second for 1KB of data
        
        E.g. 6 items / s of 4.5KB -> 6 x (5KB/1KB) (rounded up) = 30 WCUs
        
        Standard
        
        Transactional
        
        2x WCUs
        
        TransactWriteItems
        
        Reads
        
        Eventually consistent
        
        Might get stale data because of replication
        
        Strongly consistent
        
        Get correct data
        
        Higher latency
        
        Set 'ConsistentRead' parameter in API calls - GetItem
        
        Consumes 2 x RCU
        
        Read Capacity Units
        
        RCU = one SC read /s for 4KB of data
        
        E.g. 10 SC reads / s with 6KB (gets rounded to nearest 4KB) data -> 10 x 8/4 = 20 RCUs
        
        RCU = Two EC read /s for 4KB of data
        
        E.g. 16 EC reads /s with 12KB data -> 16/2 x 12/4 = 24 RCUs
        
        Transactional
        
        Consumes 2 x RCU
        
        TransactGetItems
        
        Partitions
        
        Data is stored in partitions
        
        Partition keys go through a hashing algorithm to know which partition to go to
        
        WCUs and RCUs spread evenly across partitions
        
        Throttling
        
        ProvisionedThroughputExceededException
        
        Reasons
        
        • Hot Keys: one partition is being read too many times
        
        • Hot Partitions
        
        • Very large items – RCU and WCU depend on size
        
        Solutions
        
        • Exponential backoff when exception is encountered
        
        • Distribute partition keys as much as possible
        
        • If RCU issue, we can use DynamoDB Accelerator (DAX)
        
        DynamoDB Accelerator (DAX)
        
        In-memory cache for DynamoDB
        
        Solves 'hot-key' problem
        
        TTL - 5mins
        
        Multi-AZ
        
        Min 3 nodes for prod
      - API
        
        Writing Data
        
        • PutItem
        
        o Consumes WCUs
        
        o Creates a new item or fully replace an old item (same primary key)
        
        • UpdateItem
        
        o Edits an existing item or adds a new one if it doesn’t exist
        
        o Can be used to increment Atomic Counters – number that is unconditionally incremented.
        
        • Conditional Writes
        
        o Accept a write/update/delete only if conditions are met, otherwise returns an error
        
        o Helps with concurrent access to items
        
        o No performance impact
        
        Reading Data
        
        • GetItem
        
        o Read based on primary key (Hash or hash range)
        
        o ProjectionExpression can be specified to retrieve only certain attributes
        
        o KeyConditionExpresssion
        
        o FilterExpression
        
         Use only with non-key attributes (No hash or range attributes)
        
        Returns number of items or upto 1MB
        
        Pagination
        
        • Scan
        
        o Exports the entire table
        
        o Returns up to 1MB of data – use pagination to keep on reading
        
        o Consumes lots of RCU
        
        o Limit using Limit
        
        o For faster performance use Parallel Scan
        
        • DeleteItem
        
        o Delete an individual item
        
        o Can perform a conditional delete
        
        • DeleteTable
        
        o Delete a whole table and all items
        
        o Much quicker than calling DeleteItem on all items
        
        Batch Operations
        
        BatchWriteItem
        
        o Upto 25 PutItem and/or DeleteItem in one call
        
        o Up to 16MB written, up to 400kB of data per item
        
        o Can’t update items (UpdateItem)
        
        o UnprocessedItems for failed write operations (exp backoff or add WCU)
        
        BatchGetItem
        
        o Return items from one for or more tables
        
        o Up to 100 items, up to 16MB of data
        
        o Items are retrieved in parallel to minimize latency
        
        o UnprocessedKeys for failed read operations (exp backoff or add RCU)
        
        PartiQL
        
        SQL compatible query language for DynamoDB
      - DynamoDB Streams
        
        Convert database actions to stream
        
        Send to Kinesis Data Streams
        
        Lambda via Event Source Mapping
        
        Info to write to stream
        
        • KEYS_ONLY – only key attributes
        
        • NEW_IMAGE – the entire item as it appears after it was modified
        
        • OLD_IMAGE – the entire item before modification
        
        • NEW_AND_OLD_IMAGES – both new and old images
      - TTL
        
        Defines when items will be deleted from the database
        
        Timestamp for deletion on a per item basis
        
        UC - Limit storage
      - Transactions
        
        ACID
        
        Atomicity, Consistency, Isolation, Durability (ACID)
      - Backups
        
        On-demand
        
        UC - Save a table to S3
        
        Point in time recovery
        
        Store regular back ups
  - - - Changed on command line
    - - In-flight using KMS
      - At rest using KMS
  - - - Redis - complex
      - Memcache - simple
  - - - Globally unique
      - No A, _
      - 3 -63 chars
    - - Stored in buckets / folders
      - 0 - 5TB
      - Unlimited Storage
    - - 99.9 for bucket
      - 99.99 for S3 Service
    - - 99.999999999%
    - - IA
        
        UC - Disaster Revover, Backups
      - One Zone - IA
        
        UC - Disaster Recovery
      - Intelligent Tiering
        
        Moves between tiers based on usage
        
        No retrieval costs
      - Glacier
        
        Flexible Retrieval
        
        Min Store 90days
        
        Retrieval
        
        Expedited 1-5mins
        
        Standard 3-5hrs
        
        Bulk (free) 5 -12hrs
        
        Instant Retrieval
        
        Min store 90days
        
        Deep Archive
        
        Lowest Cost
        
        Retrieval
        
        12hrs
        
        Bulk (free) 48hrs
        
        Min store 180 days
      - Standard
        
        UC
        
        Big Data Analytics
        
        Mobile / Game Apps
        
        Content Distribution
    - - Storage
      - Requests
      - Management
      - Data Transfer
      - Transfer acceleration
      - Cross-Region replication
    - - PUT
        
        Upload File
        
        200 Response
      - Multi-part upload
        
        Files > 5G
      - Select
        
        Retrieve less data through server-side filtering
      - Logging
        
        Log Bucket must be in same region
      - Pre-signed URL
        
        Users of pre-signed url inherit the permissions of the user who generated it for GET and PUt
    - - IAM Access Analyser - check access to bucket
      - S3 Analytics - access patterns
      - IAM Access Advisor - identify unused roles
      - Accounts
        
        Bucket Polices and IAM - program only
        
        Bucket ACLs and IAM - program only
        
        Cross-account IAM roles - program and console only
      - Access Points
        
        Simplify Security
        
        Own DNS
        
        Internet Origin
        
        VPC Origin
        
        Access Point Policy
      - Object Lambda
        
        Change object before it is retrieved
        
        UC
        
        Redact personal info
        
        Convert formats
        
        Resize and water mark images
    - - Bucket Policies - bucket level
      - ACL - object or bucket level
    - - Transit
        
        SSL/TLS
        
        aws:SecureTransport:false
      - Rest
        
        SSE-S3 - S3 Managed keys
        
        x-amz-server-side-encryption: AES256
        
        SSE-KMS
        
        x-amz-server-side-encryption: aws:kms
        
        SSE-C - Server Side Encryption - Customer managed keys
        
        Enforce - use bucket policy that requires the x-amz header
        
        Client-side - client encrypts data
      - CORS
        
        Enable for cross-origin requests
        
        for all origins
    - - Stores all versions - writes and deletes
      - Backup tool
      - Once enabled can only be suspended
      - Integrates with lifecycle rules
      - Enable MFA for delete
      - Required for replication
    - - Auto move objects between tiers
      - Used with versioning
      - Applied to current and previous versions
      - Delete markers - can see object but still there
    - - Cross-region
        
        compliance
        
        lower latency access
        
        cross-account replication
      - Same region
        
        log aggregration
        
        live replication between prod and test
      - Delete markers not replicated
        
        Deleting individual versions not replicated
      - Versioning must be enabled
      - Only new objects are replicated
      - No chaining of replica buckets
    - - Routes traffic through CloudFronts edge locations
    - - Key (name)
      - Value (Data)
      - Version ID
      - MetaData
        
        Can't search metadata or tags
- - - - • Git repos can be expensive
      - • Private Git Repos
      - • No size limit on repos
      - • Fully managed, highly available
      - • Code only in AWS Cloud account => increased security and compliance
      - • Security
      - • Integrated with Jenkins, AWS CodeBuild and other CI tools
    - - Recommends AWS IAM user
        
        IAM supports
        
        • SSH Keys
        
        • AWS Access Keys
        
        • Git Credentials
        
        • NOT IAM Username and Password
        
        Use Git credentials from IAM when migrating code from Github
      - Supports Git credentials
      - Cross-Account
        
        Don't share SSH keys or AWS credentials
        
        Use IAM roles and use AWS STS - Security Token Service
      - Authorisation
        
        IAM to manage user and roles
      - Authentication
        
        SSH Keys
        
        HTTPS
        
        AWS Cli Credential Helper
        
        Git Credentials for IAM
  - - - o Each stage can have sequential actions and or parallel actions
      - Manual approval at each stage
    - - Cloudwatch events - failed pipelines, cancelled stages
      - CloudTrail - audit API calls
  - - - Root of code
      - Define Env vars
      - Specify phases
      - Specify which artifacts to upload to S3
  - - - Deployment Hooks
        
        ValidateService: ValidateService is the last deployment lifecycle event. It is used to verify the deployment was completed successfully.
        
        AfterInstall - You can use this deployment lifecycle event for tasks such as configuring your application or changing file permissions
        
        ApplicationStart - You typically use this deployment lifecycle event to restart services that were stopped during ApplicationStop
        
        AllowTraffic - During this deployment lifecycle event, internet traffic is allowed to access instances after a deployment. This event is reserved for the AWS CodeDeploy agent and cannot be used to run scripts
    - - Inplace
      - Blue/Green
      - Types
        
        o AllAtOnce – most downtime
        
        o HalfAtAtime: reduced capacity by 50%
        
        o OneAtATime: slowest but lowest availability impact
        
        o Custom: define own %
      - Rollback
        
        Auto
        
        Manual
    - - Needs CodeDeploy Agent
        
        EC2 must have permission to access S3
    - - Deployment
        
        • Linear: grow traffic every N minutes unit 100%:
        
        o LambdaLinear10PercentEvery3Minutes
        
        o LambdaLinear10PercentEvery10Minutes
        
        • Canary: try X% then 100%
        
        o LambdaCanary10PercentEvery5Minutes
        
        o LambdaCanary10PercentEvery30Minutes
        
        • AllAtOnce: immediate
      - Integrated with SAM
      - Traffic shift between aliases
    - - Deploy new task definitions
      - Only Blue/Green deployments
        
        Linear
        
        Canary
        
        AllAtOnce
  - - - Retrieve dependencies
  - - - Need CodeGuru Agent
- - - - 30 Dimensions per metric
      - Timestamps
      - Dashboards
      - Every AWS Service
      - Logs never expire
      - EC2
        
        Metrics every 5mins
        
        Detailed Metrics 1 mins - Faster Auto Scaling
        
        Memory not pushed as a standard metric
        
        CloudWatch Agent
        
        CloudWatch Logs Agent
        
        Old version
        
        Can only send to CW logs
        
        CloudWatch Unified Agent
        
        Additional system level metrics: ram, processes
        
        Collect logs to send to CW logs
        
        Centralised configuration using SSM Parameter Store
        
        Need to run CloudWatch Agent to get EC2 logs
        
        Metric Filter
        
        Up to 3 dimensions
      - Custom
        
        Define own metrics
        
        APi PutMetricData
        
        Use dimensions to segment metrics
        
        Metric resolution
        
        Standard 1min
        
        High 1/5/10/30sec
    - - Log Groups
        
        Arbitrary name, representing an application
      - Log streams
        
        Within app / log files / config
      - Send Logs to
        
        S3
        
        Kinesis Data Streams
        
        Kinesis Data Firehose
        
        Lambda
        
        OpenSearch
      - Encrypted by default or KMS
      - Log Insights
        
        Query Logs
        
        Not real-time
      - S3 Export
        
        Upto 21hrs to export
        
        Api CreateExportTask
      - Subscription Filter
        
        Filters logs delivered to your destination
    - - States
        
        OK
        
        INSUFFICIENT_DATA
        
        ALARM
      - Period
        
        Time to evaluate a metric
        
        High resolution: 10 / 30 / 60 s
      - Targets
        
        EC2
        
        ASG
        
        SNS
      - Composite Alarms
        
        Monitor Multiple other alarms
        
        AND OR Conditions
        
        UC - Reduce 'Alarm Noise'
        
        Test Alarms using CLI
    - - Reproduces what customers do
      - Checks endpoints etc
      - Scripts: Node.js or Python
      - Programmatic access to a headless Google Chrome browser
      - Blueprints
        
        Checks against screenshots and more
  - - - Calls to AWS Services
      - Http calls
      - Database calls
      - Queue calls
    - - Low lever packet interceptor
      - Lambda runs daemon for you
      - Apps must have IAM rights to right data to X-Ray
    - - Segments
        
        Each app sends them
      - SubSegments
        
        More details in Segment
      - Trace
        
        Segments collected together to form end to end trace
      - Sampling
        
        Reduce requests sent to X-Ray
        
        Reduce costs
        
        Reservoir
        
        1st request per second
        
        Rate
        
        % of requests above the reservoir
      - Annotations
        
        Key value pairs used with index traces and filters
      - Metadata
        
        Key value pairs not used in indexing or searching
    - - Write
        
        PutTraceSegments
        
        Uploads Segment Docs
        
        PutTelemetryRecords
        
        Daemon to upload telemetry
        
        GetSamplingRules
        
        GetSamplingTargets
        
        GetSamplingStatisticSummaries
      - Read
        
        GetServiceGraph
        
        BatchBetTraces
        
        Gets traces specified by ID
        
        GetTraceSummaries
        
        Retrieves IDs and annotations for traces available for the timeframe
        
        GetTracedGraph
    - - Elastic Beanstalk
        
        XRayEnabled: true
      - EC2
        
        Run as daemon
      - Fargate
        
        Run as sidecar
  - - - X-Ray
      - CloudWatch
      - Prometheus
      - Datadog
  - - - Console
      - SDK
      - Cli
      - AWS Services
    - - CloudWatch Logs
      - S3 - query logs with Athena
      - Logs stored for 90 days
    - - Management Events on AWS account
      - Data Events
        
        S3 activity
        
        Lambda execution
      - Insights
        
        Inaccurate resource provisioning
        
        Hitting service limits
        
        Bursts of IAM actions
        
        Gaps in maintenance activity
        
        Creates baseline of normal management events
        
        Anomalies appear in CloudTrail
- - - - On-Demand
      - Reserved
      - Spot
      - Dedicated Hosts - same physical server
      - Dedicated Instances - own VPC, share with other EC2 from same account
      - Burstable Performance - free for 1st year
    - - Stateful - what goes in comes out
      - Any number of EC2s
      - An EC2 can have multiple SGs
    - - Deny ips
        Deny all inbound and outbound traffic
    - - Universal
        Attached and detached to running instances
    - - Multi-AZ
      - Even distribution over AZs
      - In VPC, EC2s launched in own subnets
      - Scaling Policies - 5min cooldown
        
        Dynamic - CPU, CW alarms
        
        CPU
        
        Request count per target
        
        Average Network I/O
        
        Scheduled - time of day
        
        Predictive - AI
    - - EC2 learn about themselves without an IAM role
    - - Credential Order
        
        1 Command Line
        2 Env Vars
        3 Credentials file
        4 Config file ~/.aws/config
        5 Container credentials for ECS tasks
        6 Instance profile credentials
    - - Every 5mins
      - Detailed Metrics 1min - fast auto-scaling
      - Memory not pushed as standard metric
    - - Need to run CloudWatch Agent to get EC2 logs
    - - Runs during the boot cycle when you first launch an instance
      - Runs with root privelges
  - - - ALB
        
        Http/https traffic - layer 7
        
        Supports multi-header values
        
        Use ALB Access logs to analyse incoming requests for latencies and the clients IP patterns
      - NLB
        
        TCP traffic / extreme performance - layer 4
      - Classic LB
    - - Path
      - Hostname
      - Request url
      - Source ip
      - NOT Client Location
    - - Lambdas
      - Private ips
      - EC2s
      - NOT NLBs
      - The Load Balancer generates the HTTP 503: Service unavailable error when the target groups for the load balancer have no registered targets
  - - - ELB
        
        Can't change ELB type after deployment
        
        Can change config after deployment
        
        Migration
        
        Create new env with same config but new ELB
        Deploy to new env
        Perform CNAME swap
      - ASG
      - Monitoring
      - EC2s
      - RDS
        
        Create separate RDS for Prod
        
        Provision with deployment for test
    - - Just for underlying services
    - - Mutlitple
        
        Prod
        
        Test
    - - Lots of languages
    - - Web
        
        ELB
        
        SGs
        
        ASG
        
        EC2s
        
        AZs
      - Worker
        
        SQS
        
        ASGs
        
        AZs
        
        EC2s
    - - Replaces failed instances with instances running the last successful version
      - Modes
        
        All at once
        
        Fastest
        
        Downtime
        
        No additional cost
        
        Rolling
        
        Update a few at a time
        
        Runs both versions simultaneously
        
        Slow
        
        No additional cost
        
        Immutable
        
        Spins up new instances in a new ASG then swaps
        
        Zero downtime
        
        Longest deployment
        
        Quick rollback
        
        EC2 Burst Balance Lost
        
        Blue / Green
        
        Create a new environment and switch over when ready
        
        Zero downtime
        
        Can use Route53 with weighted policies
        
        Manual process
        
        Traffic Splitting
        
        Canary testing
        
        Temporary ASG deployment
        
        Deployment health is monitored for auto rollback
        
        EC2 Burst Balance lost
    - - Deploy zip file to EB
      - .ebextensions/<myconfi>.config
      - Add resources
        
        RDS
        
        ElastiCache
        
        DynamoDB
      - Uses CloudFormation under the hood
        
        Resources deployed using config file
  - - - • Process highly sensitive data in a isolated compute environment
        
        o Personal Identifiable Information (PII), credit cards etc
      - • Fully isolated virtual machines, hardened and highly constrained
        
        o Not a container, not a persistent storage, no interactive access, no external networking
      - • Helps reduce the attack surface for sensitive data processing apps
        
        o Cryptographic Attestation – only authorized code can be running in your Enclave
        
        o Only Enclaves can access sensitive data (integration with KMS)
- - - - Use yaml to generate resources
      - • SAM – Serverless Application Model
      - • Framework for developing and deploying serverless applications
      - • All config in yaml
      - • Generate complex CloudFormation from simple SAM YAML file
      - • SAM can use CodeDeploy to deploy lambda functions
      - • SAM can help you run lambda, API gateway, DynamoDB locally
    - - • Transform Header indicates a SAM template
        o Transform: ‘AWS::Serverless-2016-10-31’
      - Supported resources
        
        AWS::Serverless::Api
        
        AWS::Serverless::Application
        
        AWS::Serverless::Function
        
        AWS::Serverless::HttpApi
        
        AWS::Serverless::LayerVersion
        
        AWS::Serverless::SimpleTable
        
        AWS::Serverless::StateMachine
      - Policy Templates
        
        Apply permissions to your lambda templates
    - - Synchronise project with SAM resources
      - Commands
        
        sam sync
        
        sync code and infrastructure
        
        sam sync --code
        
        sync code changes only - no infrastructure
        
        sam sync --code --resource AWS::Serverless::Function
        
        sync all lambda and dependencies
        
        sam sync --code --resource_id HelloWorldLambdaFunction
        
        sync specific resource by id
        
        sam sync --watch
        
        monitor file changes
    - - o Sam local start-lambda
        
        Start lambda locally
      - o Sam local invoke
        
        Invoke lambda locally
      - o Sam local start-api
        
        Start API gateway endpoint
      - o Sam local generate-event
        
        Generate events for lambda functions
    - - samconfig.toml
      - sam deploy --config-env dev
      - toml -> [dev.deploy.parameters]
  - - - Create the app from a template provided by AWS CDK -> Add code to the app to create resources within stacks -> Build the app (optional) -> Synthesize one or more stacks in the app -> Deploy stack(s) to your AWS account
    - - Use cdk synth to generate the CloudFormation Template
        
        sam local invoke -t mytemplate
    - - Layer 1
        
        Lowest - need all resources specified
      - Layer 2
        
        More about intent
      - Layer 3
        
        Bring up whole resources
    - - Provision resources for CDK deployment
    - - Use CDK assertions module
      - Fine grained assertions
        
        Property testing
      - Snapshot testing
        
        Template comparisons
  - - - Infrastructure as Code
      - Like Terraform
      - Stacks referenced by name
      - Credentials
        
        Only root users can create CF key pairs
      - Templates are loaded into S3 and then referenced by CF
      - Deleting a stack will delete all the components
      - Deployment
        
        Manual
        
        Code editor and console
        
        Automatic
        
        yaml file, and cicd
      - Termination Protection
      - Custom Resources
        
        Define resources not yet supported on CF
    - - AWSTemplateFormatVersion
        
        Id fof the template version
      - Description
        
        Comments about the template
      - Resources
        
        Mandatory
        
        AWS resources declared in template
      - Parameters
        
        Dynamic inputs for template
        
        Gets values from
        
        Systems Manager
        
        secretsmanager - secrets in Secrets Manager
        
        Parameter Store
        
        ssm - plain text
        
        ssm-secure - secure strings
        
        Secrets Manager
        
        ManageMasterPassword
        
        Creates admin secret implicity for RDS
        
        You can't put a condition in the parameters section
        
        Type safe
        
        String
        
        Number
        
        List<Number>
        
        Min/Max
        
        CommaDelimitedList
        
        Reference
        
        !Ref MyParam
        
        Declare - Parameters:
        MyParam
        
        Commands
        
        NoEcho
        
        AllowedValues
        
        Pseudo Parameters
        
        AWS:AccountId
        
        AWS::Region
        
        AWS::StackID
        
        AWS::StackName
        
        AWS::NotificationARNs
        
        AWS::NoValue
      - Mappings
        
        Static variables for your template
        
        Definition
        
        Mappings:
        
        RegionMap:
        
        us-east-1:
        
        HVM64: ami-0ff8a91507f77f867
        
        Referencing
        
        ImageId: !FindInMap [RegionMap, !Ref "AWS::Region", HVM64]
        
        Good to know when you have details in advance
      - Outputs
        
        References to what has been created
        
        Optional
        
        Output values can be imported into other stacks
        
        View outputs in CLI or Console
        
        Exported Outputs must have unique names within a single region
        
        Example
        
        Outputs:
        BackupLB:
        Description:
        DNS name of LB
        Value: !GetAtt BackupLoadBalancer.DNSName
        Condition: CreateProdResources
      - Conditions
        
        List of conditions for resource creation
        
        Uses the parameters
        
        Example
        
        Condition:
        isProduction: !Equals [ !Ref environment, production]
    - - Fn::Ref
        
        get params or resources NOT Conditions
        
        !Ref MyVPC
      - Fn::GetAtt
        
        Get attribute
        
        !GetAtt Ec2Instance.AvailabilityZone
      - Fn::FindInMap
      - Fn::ImportValue
      - Fn::Base64
      - Conditional Functions
        
        Fn::If
        
        Fn::Not
        
        Fn::Equals
    - - Default
        
        All rolls back
      - Can disable and investigate
      - Manual
        
        Fix manually then issue ConintueUpdateRollback API from Console
    - - CAPABILITY_NAMED_IAM
        
        If resources are named
      - CAPABILITY_IAM
        
        Enable CF to update IAM resources: roles, groups, polices
      - InsufficientCapabilitiesException
        
        Capability hasn't been acknowledged before deployment
    - - Default
        
        Delete
      - Retain
        
        Individual resources
    - - Default
        
        Updates are allowed on all resources
      - Can set different policies for different resources
    - - Single Template
        
        Update/Create/Delete Stacks across multiple accounts and regions
        
        Target accounts
      - All associated stack instances are updated at the same time
      - Can be applied into all accounts
      - Only Admin can create stacksets
- - - - Provision and maintain EC2s
      - Ec2 must run the ECS Agent
        
        IAM Role to access
        
        ECS
        
        Cloudwatch Logs
        
        Pull image from ECR
        
        Access Secrets Manager
    - - Serverless
      - Create Task definitions
    - - Scale increase tasks
      - Each task has task role to allow access to different AWS services
      - Task Role is defined in the task definition
      - Placement Strategy
        
        Binpack
        
        Stuff them into one instance
        
        Random
        
        Spread
        
        Spread over AZs etc
        
        Mix and Match
        
        Contraints
        
        distintInstance
        
        each task on a different container instance
        
        memberOf
        
        place task besed on CQL (Cluster Query Language)
        expression
    - - Serverless
      - Mount onto ECS tasks
      - Good for both launch types
      - Tasks in any AZ share file store
      - UC - Multi-AZ shared storage for containers
      - File Storage for
        
        EC2
        
        Containers
        
        On-prem
    - - Calls to other AWS Services
      - Enable ECS_ENABLE_TASK_IAM_ROLE in the /etc/ecs/ecs.config file
    - - Cloudwatch Metric
        
        Action in ALARM state
        
        CPU Utilisation
        
        Memory Utilisation
        
        ALB Request count per target
      - Min or Max instances never exceeded with EC2s
      - Scaling Types
        
        Step Scaling
        
        Based on Cloudwatch alarm
        
        Scheduled Scaling - data / time
        
        ECS AutoScaling
        
        Fargate Auto Scaling
      - Host port = 0 (or empty)
        
        Launch multiple containers on the same instance
    - - Cli tool to release prod ready containerised apps
  - - - Vulnerability Scanning
      - Versioning
  - - - EC2
      - Fargate
    - - Managed
      - Self-Managed
      - Fargate
- - - - Fully Managed
      - Used to Decouple Apps
      - Unlimited messages and throughput
      - Retention
        
        4 - 14 days
      - Message Size
        
        < 256kb
        
        Can use extend client for larger messages
        
        Point to location on S3 bucket
      - Duplicate messages
      - Ordering
        
        Out of Order
      - Producers
        
        SendMessage using SDK
      - Consumers
        
        Poll for messages
        
        EC2 or Lambda
        
        Receive up to 10 per time
        
        DeleteMessage after processing
        
        ASG
        
        Backlog per instance metric to scale
        
        Message Visibility Timeout
        
        After msg is polled how long before it becomes visible to other consumers
        
        ChangeMessageVisibility Api
        
        Default 30s
        
        DLQ
        
        MaxReceives goes onto DLQ
        
        DLQ for FIFO queue must also be a FIFO queue
        
        DLQ for Standard queue must also be a Standard queue
        
        Delay Queue
        
        Consumers don't need the message immediately
        
        Long Polling
        
        If no msgs then it will wait for some to arrive - 1 - 20sec wait times
    - - Decouple application between tiers
      - Used with SNS to create fan out
    - - Https
      - KMS
      - Client-side
      - Access
        
        IAM policies
        
        Access to SQS
        
        SQS Access policies
        
        Cross account access
    - - Limited throughput
        
        300msgs/s or 3000msgs/s with batching
      - Ordered queue
      - Messages are processed in order
      - Create queue with .fifo
      - Deduplication
        
        Interval
        
        5mins
        
        Methods
        
        Content-Based
        
        sha256 of message body
        
        Explicit
        
        Message Deduplication ID
        
        One consumer per group ID
      - Message Grouping
        
        One consumer for message group id
        
        Ordering across groups isn't guarenteed
    - - CreateQueue
        
        Can't change queue type after creation
      - DeleteQueue
        
        Delete queue after use
      - PurgeQueue
        
        Delete all messages in the queue
      - SendMessage
      - ReceiveMessage
      - DeleteMessage
      - MaxNumberOfMessages
        
        1 - 10
      - ReceiveMessageWaitTimeSeconds
        
        Long Polling
      - ChangeMessageVisibility
      - Batch
        
        SendMessage
        
        DeleteMessage
        
        ChangeMessageVisibility
  - - - Send one message to many receivers
      - Pub/Sub Model
      - Upto 12_500_000 subs per topic
      - 10_000 topic limit
      - Push Data
    - - Https
      - KMS
      - Client-side
      - IAM Policies for access to SNS
      - SNS Access Policies
        
        Cross account access
        
        Allow other services to write to SNS
    - - SNS + SQS Fanout
      - S3 events to SNS -> SQS so multiple actions for single S3 event
    - - Messages with same group ID ordered
      - Deduplication
        
        Content based
        
        Deduplication ID
      - Subscribers
        
        SQS Standard
        
        SQS FIFO
      - Limited throughput
    - - Json policy
      - If subscription doesn't have a policy it will receive every message
    - - SQS
      - Lambda
      - Https Endpoint
      - Kinesis Data Firehose
        
        Not Kinesis Data Streams
  - - - Real-time Streaming
      - Collect, process and analyse data in real-time
    - - Kinesis Data Streams
        
        Capture, process and store data streams
        
        Attributes
        
        Retention 1 - 365 days
        
        Replay
        
        Data can't be deleted
        
        Data with same partition goes to same shard
        
        Producers
        
        SDK
        
        Kinesis Producer Library (KPL)
        
        Kinesis Agent
        
        Data record
        
        Sequence number
        
        Partition key
        
        Data blob
        
        API
        
        PutRecord
        
        Batching
        
        PutRecords
        
        1 more item...
        
        Erros
        
        ProvisionedThrougputExceeded
        
        Shard gets overloaded
        
        Solution
        
        2 more items...
        
        Consumers
        
        Own code
        
        Kinesis Client Library (KCL)
        
        Each shard read by only one KCL instance
        
        Progress checkpointed by dynamoDb
        
        Runs on
        
        EC2
        
        Elastic Beanstalk
        
        On-prem
        
        Records read in order at shard level
        
        SDK
        
        Lambda
        
        Supports
        
        Classic Fan-out
        
        Enhanced fan-out
        
        Retries is error
        
        Process up to 10 batches per shard simultaneously
        
        Kinesis Data Firehose
        
        Custom
        
        Shared (Classic) Fan-out Consumer
        
        Pull
        
        Low number of consumers
        
        GetRecords
        
        Consumers poll using GetRecords Api
        
        2 more items...
        
        Latency
        
        1 more item...
        
        Lower cost
        
        Enhanced Fan-out
        
        Push
        
        Multiple apps for same stream
        
        2MB /s per consumer per shard
        
        Latency
        
        1 more item...
        
        Higher cost
        
        Pushes data to subscribers of Http/2
        
        1 more item...
        
        Limit of 5 consumer apps per data stream
        
        Capacity Modes
        
        Provisioned
        
        Choose number of shards
        
        Shard gets 1MB/S or 1000 records /s
        
        Pay
        
        Stream per hour
        
        Data in/out per GB
        
        Shard per hour
        
        On-demand
        
        Don't need to provision or manage capacity
        
        4MB /s or 4000 records /s
        
        Scales automatically based on last 30day throughput
        
        Shards
        
        Each shard can only be read by one app
        
        Split to increase throughput of hot shard
        
        old shard closed once data has expired
        
        Merging
        
        Reduce capacity
        
        Save costs
        
        Not more than two shards can merge at once
      - Kinesis Data Firehose
        
        load data streams into AWS data stores
        
        Fully managed
        
        Auto scaling
        
        Serverless
        
        Write to
        
        Redshift
        
        S3
        
        OpenSearch
        
        3rd Parties
        
        Splunk
        
        MongoDb
        
        DataDog
        
        Custom
        
        Http endpoint
        
        Pay
        
        Data going through firehose
        
        Near Real-time
        
        Buffer
        
        0 - 900s
        
        Size 1MB
        
        Data
        
        Supports most formats
        
        Transformations using lambda
        
        Failed data or all data to S3
        
        No data storage
        
        No replay
      - Kinesis Data Analytics
        
        Analyse data streams with SQL and Apache Flink
        
        Analytics on
        
        Kinesis Data Streams
        
        Kinesis Firehose
        
        Using SQL
        
        Fully managed
        
        Auto Scaling
        
        Pay for consumption rate
        
        UC
        
        Time-series analysis
        
        Real-time dashboards
        
        Real-time metrics
      - Kinesis Video Streams
        
        Capture, process and store video streams
- - - - A Records - maps to an ip
      - AAAA - maps to ipv6
      - CNAME - maps to another domain name
      - Alias - maps to AWS resource
      - SOA - Start of Authority
    - - Simple
      - Weighted
      - Latency-based
      - Failover
      - Geoproximity - Route53 decides based on location of users and resources
      - Multivalue
      - Geolocation - location of users
      - Ip based - specify CIDR
  - - - Separate to region
    - - Where the files are accessed from
      - ALB
      - EC2
      - Static Website
        
        UC - Static content - available everywhere
      - Multiple Origin
        
        Route to different origin types based on path
      - Origin Groups
        
        UC - Incase of failover
    - - Web
        
        Websites and images
      - RTMP
        
        Media Streaming
    - - Charged for clearing cache
        
        Cloudfront invalidation - pick up new contents
      - Can't cache based on http methods
      - Cache Key
        
        hostname
        
        resource part of url
        
        Cache Policy
        
        Headers
        
        Cookies
        
        Query Strings
        
        Everything included in the cache key are included in origin requests
    - - CF has public key
      - Private key used to sign part of url
      - Signed URL
        
        Access to individual files
      - Signed Cookie
        
        Access multiple files
      - Root user can have up to two active CF key pairs
    - - Allow list
      - Blocklist
      - UC - Copyright laws
      - Country == Geo IP Database
    - - All
        
        All regions
      - 200
        
        excludes most expensive regions
      - 100
        
        Least expensive regions
    - - Protect sensitive fields
      - Asymmetric encryption
      - Up to 10 fields in POST request
    - - Monitor delivery performance
        
        Kinesis data streams
    - - Execution time <1ms
      - Max memory 2MB
      - Total Pakdage size 10kB
      - No access to request body
      - No access to file system
      - Millions of requests per second
      - JavaScript only