Please enable JavaScript.

Coggle requires JavaScript to display documents.

AWS (Security, Identity, Compliance (IAM (Secret Token Service (STS)…

- - - - Access Key ID
      - Secret Access Key
    - - Allows user to login to AWS console without assigning IAM credentials
      - Limited privilege credentials for IAM users
      - users that you authenticate
        (federated users from an on-premise directory)
      - Federation (AD) uses SAML 2.0
- - - - Multi-AZ deployments
        
        Cannot connect to standby instances
    - - Authenticate RDS using IAM user roles
  - - - HTTP 400 Bad request
      - ProvisionedThroughputExceeded Exception
    - - Auto Scale up when needed
  - - - Single database to span multiple regions
      - replicate your data with no impact on performance
      - fast local read
      - secondary region can be promoted to be primary
        in-case of DR
    - - Handled Automatically
      - Aurora Replica
        
        Flip CNAME to point to a healthy replica
      - Aurora
- - - - Cloud Data Warehouse
      - Query & Export data to|from data lake
      - Federated Query
  - - - Data preparation
      - Extract, transform, load (ETL)
      - Central metadata repository - AWS Glue Data Catalog
- - - - Will give a specific task to only one worker, or decider
      - At most One decision task outstanding at a time for workflow execution
  - - - Have all capabilities of the standard queue
      - First-In-First-Out Delivery
        Ensures the order of messages
      - Exactly-Once Processing
        Ensures messages are not delivered multiple times
    - - Unlimited Throughput
      - At-Least-Once Delivery
      - Best-Effort Ordering
- - - - Standard
      - Standard-IA
      - Standard OneZone-IA
      - Glacier
      - Glacier Deep Archive
      - S3 Intelligent Tiering
    - - Min. 30 days to be able to transfer Storage class
      - Can apply to either current version, or previous versions
    - - Use separate Prefix to increase performance
      - Configure prefix as same as domain if serving objects from domain
    - - Enable Versioning to prevent accidental deletion
      - Delete Version will put a deletion marker as a Soft-Delete
      - Enable MFA authentication to prevent accidental deletion
      - Replication will not replicate version that was created before replication enabled
      - Delete marker can be removed to restore object
      - Cannot be disabled
        
        Delete the versioning bucket
        
        Suspend Versioning to stop version grow
    - - Encryption At-Rest
        
        Server Side Encryption - S3 (SSE-S3)
        
        Server Side Encryption - Customized Key (SSE-C)
        
        Manage The Encryption Key
        
        Server Side Encryption - KMS (SSE-KMS)
        
        Upload file calls GenerateDataKey API
        
        Download file calls Decrypt API
        
        Upload/Download costs KMS quota (region specic)
        
        Quota
        
        Resource quotas: Limit the number of each type of AWS KMS resource
        
        Request quotas: Limit the number of requests for AWS KMS API operation in a specific interval
        
        Encrypt data before Send to S3
        (Client-Side Encryption)
      - Encryption In-Transit
        
        SSL/TLS
    - - S3:ObjectCreated:*
      - S3:ObjectCreated:Put/Post/Copy
      - Max single PUT limit to 5GB
      - Can enable Notification to trigger upon Event happened
        
        SNS Topic
        
        SQS Queue
        
        AWS Lambda
      - S3:ObjectRemoved:DeleteMarkerCreated
        (When marking a version as delete)
      - Data over 5GB should use multi-part upload
        Recommanded: 100MB ~ 5GB
    - - Private by default
      - "Make public" will make the latest version public
      - Object Lock
        
        WORM Drive (Write Once Read Many)
        
        Can apply to any object or bucket
        
        Governance mode
        
        Can't alter/delete object/version or alter object's lock setting
        Unless if user has special permission
        
        Compliance Mode
        
        Retention Period applied
        
        Object version can't be overwritten or deleted for the Retention period
        
        protected object/version cannot be overwritten by any user
        (including root)
        
        Legal Holds
        
        Remains effective until removed
        
        s3:PutObjectLegalHold permission granted user can place Legal hold
        
        Glacier Vault lock
        
        WORM Policy
        
        Vault lock policy
        
        lock the policy to prevent future edit
    - - Byte-Range Fetches
        Multi-part download that downloads partial amount of file
      - Amazon S3 Transfer Acceleration
        (Similar to Global Accelerator)
    - - SQL - like search
      - Retrieve only a subset of an object
      - Get data by rows/columns (CSV) using simple SQL query
      - Save Cost in Data Transfer
    - - Bucket Policy & IAM
        
        Applies to Entire bucket
        
        Programmatic Access Only
      - Bucket ACL & IAM
        
        Individual Object
        
        Programmatic Access Only
      - Cross Account IAM Roles
        
        AWS Console & Programmatic Access
        
        Steps
        
        Using Master Account, Create Role &
        Select type of trusted entity
        
        Select Another AWS Account
        Allowing this user to access S3
        
        Attach Policies to the Role we Created
        S3 admin maybe
        
        Give link to user who can switch role in console
    - - S3 Bucket Acts as Origin where we want to distribute content
    - - Versioning must be enabled
      - Happens to objects changed after Replication is setup
      - Only replicate Newest Version
        
        Contact AWS support Center to
        "Replicate Existing Object"
      - Can be configured to replicate to single or multiple buckets
        
        Require dest. bucket(s)
        
        AWS Identity And Access Management (IAM) Role
        that S3 can assume to replicate object on your behalf
      - Cross Account, Cross Region
      - Dest. bucket(s) must enable object locks if source does
    - - Single Object up-to 5TB
      - Unlimited Storage
      - 1 ~ millions of connection over web
    - - CloudTrail
        Audit all Amazon S3 bucket access by AWS user, role or service
      - Server Access Logging
        
        detailed record for the requests that are made to a bucket
        
        Enable per bucket
        
        Can deliver log to another bucket using log delivery
  - - - EFS-IA for infrequent Access
      - UseCase: Move files to EFS-IA not accessed for 30 days to Standard class
      - 7, 14, 30, 60 days to trigger policy
    - - Network File System (NFS) 4 protocol
      - Can be accessed by multiple EC2 instances (Linux)
      - Stores Data & Metadata across multiple AZ in a Region
      - Strong data consistency and file locking
      - Allows file access control via POSIX
      - Uses DataSync to move files In/Out
      - Can Schedule Auto-incremental backup
      - Supports Encryption in-transit & at-rest
      - File System created in AWS Console are automatically backed-up for 35 days using AWS Backup
    - - General Purpose (default)
      - MAX I/O
        Scarifies latency of file operation for high level throughput
    - - Bursting Throughput mode (default)
      - Provisioned Throughput mode
        
        Single AZ, no redundancy
        
        Single EC2 in single AZ
    - - Create Mount Target(s) in VPC
      - provides an IPv4 NFSv4 mount point
      - 1 mount target in each AZ in a Region
      - Mount filesystem using DNS Name
        will resolve to IP address
    - - IAM
      - Enforce an OS user/group and directory for file system requests
    - - open-after-close consistency semantics
      - Write operations will be durably stored across AZ
      - read-after-write consistency for Application performs Sync data access
    - - CloudWatch alarms
      - CloudWatch logs
      - CloudWatch Events
      - CloudTrail log monitoring
      - Log files on filesystem
  - - - Fully Managed MS Windows file system
      - Supports SMB, NTFS and Windows AD
      - Features
        
        Multiple users file system
        
        Works with Amazon Services
        
        EC2
        
        Workspace instances
        
        AppStream 2.0 instances
        
        VM running VMWare Cloud on AWS
      - Use Cases
        
        Work with Windows AD
        
        Access FSx file system using Direct Connect
        
        MS Windows File Share
    - - Popular open-source parallel system
      - SSD || HDD
      - Use Lustre as Hot storage, and use S3 as Cold storage
      - Can link to S3 & transparently present S3 object as files and allows you to write result back to S3
      - POSIX-compliant file system interface
      - HA & Durability
        
        Auto replicates your data within an AZ in a Region
        
        Single / Multiple AZ deployment
        
        Volume Shallow copy to perform daily backup to S3
        
        Support Restoring individual file(s)
        
        Backup retains 7 days as default
  - - - General Purpose SSD (gp2, gp3)
        
        SSD
        
        3 IOPS per GB, up to 16,000 IOPS
        
        1~16 TB
      - Provisioned IOPS (io1, io2)
        
        50 IOPS ~ 64,000 IOPS per volume
        
        4~16TB
        
        SSD
        
        100 GiB volume can be provisioned with up to 5,000 IOPS
        
        IOPs to Volume ratio => 50:1
      - Throughput Optimized HDD (st1)
        
        500 IOPs per volume
        
        NO SLA for IOPs
      - Cold HDD (sc1)
        
        250 IOPs per volume
        
        No SLA for IOPs
      - Magnetic (HDD)
- - - - Connects VPC to VPC
      - Consolidate the AWS VPC routing configuration for a region
      - Hub-and-spoke architecture
      - Spoke VPC connect to the Transit Gateway to gain access to other connected VPC
      - Can peer Transit Gateway in another Region
    - - Connection between two VPCs
      - Enables routing using each VPC's private IP addresses
      - Supports Cross Region peering
      - Supports Inter-Region peering
    - - Enables connection to AWS services hosted by other AWS accounts
        (endpoint services)
      - Connect to some AWS Services
      - Endpoint created in your VPC
        VPC Endpoints
        
        Entry point in your VPC that enables you to connect privately to a AWS service
        
        Types
        
        Gateway endpoint
        
        Specify VPC to deploy Endpoint & service you are connecting (Name, ID and region)
        
        Attach an endpoint policy to end point that allows access to some or all of the service to which your are connecting.
        
        Specify one or more route tables in which to create routes to the service.
        
        Interface endpoint
        
        Choose VPC to create endpoint
        
        Choose a subnet to use the interface endpoint.
        Create an Endpoint Network Interface in the subnet
        
        Specify the security groups
        
        Gateway Load Balancer endpoint
        
        Endpoint service
        The services in your VPC being connected to by other Enpoints
      - Manage access to endpoints using VPC Security Groups
    - - Features
        
        Cross-zone Load balancing
        
        Ensures traffic is distributed evenly between Available instances in ALL AZ
        
        SSL Offloading
        
        Distribute traffic across instances in single AZ or multiple AZ
        
        Internet facing
        
        Node have public IPs
        
        Route traffic to the private IP addresses of EC2 instances
        
        Need one public subnet in each AZ where ELB is defined
        
        inbound Security Group: Source set to /0 any address
        
        outbound Security Group: Destination set to the EC2 Registered instance security group
        
        Internal only
        
        Have private IP
        
        Route to private IP
        
        Dont' need Internet Gateway
        
        inbound Security Group: Source set to VPC CIDR
        
        outbound Security Group: Destination set to the EC2 Registered instance security group
        
        ELB Nodes Consumes IP address
        
        Ensure at least /27 subnet
        
        at least 8 IP address available in order for ELB to Scale
      - Network Load Balancer (NLB)
        
        Operates at the connection level
        
        layer 4
        
        high performance, low latency and TLS offloading at scale
        
        Can have Static/Elastic IP
        
        Supports UDP and static IP addresses as targets
      - Application Load Balancer (ALB)
        
        Operates at the request level
        
        Layer 7
        
        Routing
        
        Path-based
        
        Host-based
        
        Query string parameter-based routing
        
        Source-ip address based routing
        
        Sticky Sessions
      - Classic Load Balancer
        
        Sticky sessions
      - Security Groups
        
        Need to allow ports and protocols for the health check port and backend listeners
        
        Inter
  - - - Can Migrate IP address as fixed entry point
      - Migrate up to /24 Ipv4 addresses
      - Choose up to /32 IP addresses when creating accelerator
- - - - Monitor tool for AWS resources and Apps
      - Display metrics and create Alarms that watch the metrics
        Sends notification or automatically make changes to the resources being monitored, when threshold is breached,
      - Metrics repository
        
        basic metrics added by AWS services
        
        Customized metrics can be added to repo.
      - Metrics are separated between regions
    - - Namespaces
        
        Container for metrics
        
        No default namespaces
        
        AWS namespace format: AWS/service
      - Metrics
        
        Time-oriented set of data published to Cloudwatch
        
        Region Specific
        
        Cannot be deleted, expires itself after 15 months of no new data is published to them.
        
        timestamp is required, can be value of +- 14 days
        If no timestamp is provided, CloudWatch will create it for you
        
        Enable detail-monitoring for several services to collect metrics
        
        Metric match enables you to query multiple CloudWatch Metrics and use new time series based on these metrics
        
        CloudWatch DOES NOT collect Memory utilization & Disk Space Usage metrics from EC2 instances
        You need to install CloudWatch Agent on instance
      - Dimensions
        
        Max. of 10 dimensions to a metric
        
        by adding a unique dimension to a metric, you are creating new version of the metric
      - Statistics
    - - Watch a single metric for a specified period of time
        Perform 1 or more actions based on the value of metric relative to a threshold over time
      - Alarm for monitoring CPU and load balancer latency. for managing instances and for billing alarms.
      - invokes actions for sustained state changes only.
      - States
        
        OK - The metric or expression is within the defined threshold
        
        ALARM - The metric or expression is outside of the defined threshold.
        
        INSUFFICIENT DATA - The alarm has just started. the metric is not available, or not enough data is available for the metric to determine the alarm state.
      - Creating ALARM
        
        Period
        
        Evaluation Period
        
        Datapoints to Alarm
    - - Customizable home page inthe CloudWatch console
      - no limit on the number of CloudWatch dashboards you can create
    - - Near real-time stream of system events that describe changes in AWS resources
      - Respond to these operational changes and take corrective action as necessary, by sending message to respond to the environment, activating functions, making changes and capturing state information.
      - Concepts
        
        Events
        
        Targets
        
        Rules
    - - Features
        
        Monitor logs from EC2 instances in real-time
        
        Monitor CloudTrail logged events
        
        default never expire
        
        Archive log data
        
        Log Route 53 DNS queries
      - CloudWatch Logs Insights
        enables you to interactively search and analyze log data in CloudWatch Logs using queries
      - CloudWatch Vended logs
        
        VPC Flow Logs
    - - Collect more logs and system level metrics from EC2 instances and on-premises servers
      - Install on instances
  - - - Performed when number of instances across AZ is not balanced
      - launching new EC2 instances in the AZ that has fewer instances
      - Terminating instances in AZ that had more instances
    - - An instance is launched
      - An instance is terminated
      - An instance fails to launch
      - An instance fails to terminate
    - - Amazon EC2
        
        Concepts
        
        Horizontal scaling (scale-out)
        
        Region specific service
        
        Spans multiple AZ within same region
        
        Free
        
        Can specify subnet to launch EC2 instances to
        
        Can Attach ELB to existing ASG, but ELB must be in same Region
        
        Can attach 1 or more Target Groups to ASG to include instances behind ALB
        
        Launch Configuration
        
        Versioned
        
        Can be edited after defined
        
        Can be used by multiple ASG
        
        Not Recommanded
        
        Scaling Group
        
        Scaling type
        
        Scheduled Scaling
        
        Dynamic Scaling
        
        Predictive Scaling
        
        Launch Template
        
        key pair
        
        AMI IDs
        
        instance type
        
        security groups
        
        parameters used to launch EC2
      - Amazon ECS
      - Amazon DynamoDB
      - Amazon Aurora
- - - - Use AWS Schema Conversion Tool (SCT) to extract data into Snowball Edge
      - Ship the Edge device back to AWS
      - Edge loads data into S3
      - AWS DMS takes the data from S3 and loads into target Data Store
  - - - Network File System (NFS)
      - Server Message Block (SMB) file servers
      - self-managed object storage
      - AWS Snowcone
      - Amazon S3 buckets
      - Amazon FSx for Windows File Server
      - Amazon EFS file systems
        (Install Agent on EC2 that connects to EFS)
    - - Deploy an Agent
        Hypervisor image to be deployed on-premises or
        AMI to be deployed on AWS
      - Create Data Transfer task
        Source & Dist. & schedule ... etc
      - Start the transfer task, monitor on AWS Console or Cloudwatch
    - - Use DataSync to Migrate files on AWS then Use FileGateway to retain access to files on AWS from on-premises file-based application
      - Use DataSync for online transfer, Snowball for offline transfer data into AWS
      - Use S3 Transfer Acceleration if already integrated with S3
      - On-Premises -> EFS -> EFS Lifecycle management -> IA storage class
    - - Max. number of tasks: 100
      - Max. number of files per task: 50 million
      - MAx. throughput per task: 10Gbps
- - - - DNS resolution & hostname depends on VPC setting
        Private VPC won't have this turned on by default
    - - Cost-effective
        
        Reserved instances
        
        Commitment of 1~n years term of usage
        
        Not suitable for periodically processes
        
        Spot Instances
        
        Use this if your application can handle shutdown or gracefully recover
        
        Can be interrupted when Spot price exceeds max. price set
      - Dedicated instances
      - On-Demand instances
    - - [Restart] -> Pending -> terminated
        
        An EBS snapshot is corrupt
        
        Reached EBS volume limit
        
        Root EBS maybe encrypted and you do not have permission to access KMS to decrypt
        
        Instance store-backed AMI used to launch the instance is missing a required part (image.part.xxx file)
- - - - Password
      - Database Strings
      - AM IDs
      - license codes