Please enable JavaScript.

Coggle requires JavaScript to display documents.

Tutorial (Chef (HOW IT WORKS? (If cookbooks or settings are changed then…

- - - - Leader Node - manages client connections and receives queries
      - Compute Node - store data and perform queries and computations
  - - - Automated Backups are enabled by default.
      - The backup data is stored in S3 annd we get free storage space equal to the size of our database
    - - They are done manually.
      - They are stored even after you delete the original RDS instance, unlike automated backups.
- - - - Standard Queue (default)
        
        Nearly-unlimited number of transactions per second
        
        Guarantee that a message is delivered at least once
        
        Occasionally more than one copy of a message might be delivered out of order (They do not have order)
      - FIFO Queue
        
        Upgrade on Standard Queue
        
        The order in which messages are sent and received is strictly preserved
        
        Message is delivered once and remains available until a consumer processes and deletes it
        
        Duplicates are not introduced into the queue
        
        Limited to 300 transactions per second
  - - - Kinesis Streams
        
        Producers send data in
        
        Streams store data.
        
        Data is stored in Shards (for 24h default)
        
        Then we have the Consumers which are EC2 instances that take the data and do something useful with it
        
        Once the Consumers do the calculations they send data to variety of AWS services (DynamoDB, S3, EMR, Redshift etc).
      - Kinesis Firehouse
        
        When the data comes in it is either analysed using Lambda or sent directly to S3
        
        A lot more automated and no need to worry about Shards management
      - Kinesis Analytics
- - - - Bucket - a folder in the cloud
      - Place where we can group our files
      - You can have folders inside a bucket.
        Difference?
        S3 is a universal namespace, so the buckets names must be unique globally.
      - It generates a domain name when we setup a bucket
    - - After we create a new file we can immediately view and read it.
      - When we update, it could not be updated immediately/take some time to propagate.
    - - S3 Standard
        99.99% availability,
        99.9999999% durability
        Stored across multiple devices in multiple facilities.
      - S3 - IA (Infrequently Accesses)
        For data that is accessed less frequently, bur requires rapid access when needed
      - S3 One Zone - IA
        lower-cost option for infrequently accessed data, but do not require the multiple availability zones data resilience.
      - Glacier
        Very cheap, but used for archival only.
        Different models with different retrieval times.
    - - For Storage (per Gigabyte)
      - For Requests (per request)
      - For Storage Management (essentially metadata)
      - For Data Transfer (from one region to another)
      - For Transfer Acceleration - to enable fast, easy and secure transfers of files over long distances.
        Take advantage of Amazon CloudFront's globally distributed edge locations.
    - - Rules can be added using the lifecycle tab in a bucket, to store files to Glacier after 60 days, for example.
      - Also moving between storage classes
      - Store copies of our previous version and our current version
      - Can be used to permanently delete objects as well.
      - Used to minimize storage cost
  - - - On Demand
        
        Perfect for low cost and flexibility of Amazon without up-front payment.
        
        Application with short term, unpredictable workloads that cannot be interrupted
        
        Applications being developed or tested on EC2
        
        Learning
      - Reserved
        
        Applications with steady state or predictable usage.
        
        Apps that require reserved capacity
        
        3 year contract and you will get big discount
      - Spot
        
        Apps that have flexible start and end times
        
        Applications that are only feasible at very low compute prices
        
        Users with urgent need of additional computing capacity
      - Dedicated Hosts
        
        Useful for regulatory requirements
        
        Greate for licensing which does not support multi-tenancy or cloud deployments
    - - I3 (High Speed Storage)
        
        For: NoSQL DBs, Data Warehousing, etc.
      - H1 (High Disk Throughput)
        
        MapReduce-based workloads
        
        distributed file systems (HDFS, MapR-FS)
      - T2 (Lower Cost, General Purpose)
        
        Web Servers
        
        Small DBs
      - M5 (General Purpose)
        
        Application Servers
      - C5 (Compute Optimized)
        
        CPU Intensive Apps/DBs
      - X1 (Memory Optimized = Huge amount of RAM)
        
        SAP HANA
        
        Apache Spark
      - F1 (Field Programmable Gate Array)
        
        Genomics research
        
        Financial analytics
        
        real-time video processing
        
        big data
    - - Allows us to create storage volumes and attach them to Amazon EC2 instances.
      - Once, attached, we can create a file system on top of these volumes, run database, install apps etc.
      - They are placed in a specific Availability Zone, where they are automatically replicated to protect against failures.
      - Types
        
        General Purpose SSD (GP2)
        
        Provisioned IOPS SSD - for intensive applications such as large relational or NoSQL databases
        
        Throughput Optimized HDD (ST1) - for big data, data warehouses, log processing
        
        Cold HDD (SC1) - for storage/less frequently accessed
      - EBS backed instances can be stopped. And we will not lose the data on this instance if it stopped.
      - Both EBS and Instance Store can be reboot and we will not lose data.
      - EBS Snapshots are backed up to S3 in Incremental manner.
      - First we need to deregister the AMI before being able to delete the snapshot of EBS Volume that is used as the root device.
    - - Security Groups is a Virtual Firewall
      - Instance can have multiple security groups.
      - Essentially first layer of security
      - They are used to allow certain type of traffic.
        By default all is blocked and we just allow different types of traffic.
      - All Inbound Traffic is blocked by default.
        All Outbound Traffic is allowed.
        If you create an inbound rule allowing traffic in, that traffic is automatically allowed back out again.
      - You cannot block specific IP address using Security Groups, instead use Network Access Control Lists.
    - - In EBS you can stop the instance in Instance store you cannot stop it.
      - With EBS you can stop and move your instance around, if Instance store fails then we lose it
      - Instance store created from templates in S3
      - Both can be rebooted and we will not lose data.
    - - We can create dashboard with selection of weidgets to suit our monitoring needs
      - If a metric is not in the default widgets, then
        theres is the posibility to make custom widgets.
        This requires writing some code and running it on EC2 instance to force it to report back.
      - Standard Monitoring = 5 minutes
        Detailed Monitoring = 1 minute
      - We can set Alarms that notify when particular thresholds are hit.
      - Events - CloudWatch Events helps to respond to state changes in AWS resources.
      - Logs - CloudWatch Logs help to monitor and store logs.
        That is done by installing agent on EC2 server
      - We can send data to CloudWatch from external sources also!
    - - Load Balancers and Autoscalling
        
        We need to use the DNS name for the elastic load balancers. We are never given an IP address
        
        Autoscalling
        
        Before creating Autoscalling group, we need to have launch configuration
        
        Example for usage:
        When my CPU Utilization becomes above 90% take some action.
        Action here could be: "Add 1 instance"
        The same thing could be done with removing instances.
        Essentially our instances can grow and shrink with the CPU usage.
        
        Notifications can be sent whenever instances are added, removed, failed to start etc.
        
        Load Balancers are still delivering traffic to the instances that are alive. It is figuring this out through the health checks.
        
        Then when the AutoScalling Groups realize that some instances are down it starts to launch new ones.
        
        Instances monitored by ELB are reported as: InService or OutofService
        When EC2 instance fails its healthchecks from the Load Balancer, it will stop sending traffic to it.
      - EC2 Placement Groups
        
        Clustered Placement Group
        Grouping of instances within a single Availability Zone.
        Placement groups are recommended for applications that need low network latency, high network throughput, or both.
        Only certain instances can be launched into a Clustered Placement Group (usually ones with very high RAM, CPU utilization, etc.)
        
        Spread Placement Group
        Group of instances that are each placed on distinct underlying hardware.
        Spread placement groups are recommended for applications that have a small number of critical instances that should be kept separate from each other.
        This is for multiple availability zones.
        
        AWS recommends homogenous instances within placement groups.
      - EFS (Elastic File System)
        File storage service for Amazon EC2 instances.
        EFS's storage capacity is elastic, growing and shrinking automatically as you add and remove files, so our applications have the storage they need when they need it.
        
        Only pay for the storage we use - 30cents per GB
        
        Can support thousands of concurrent NFS connections
        
        No durabillity yet.
        
        It is block-based storage
        
        We can have multiple EC2 instances connected to same EFS volume
  - - - Event-diven compute service where AWS Lambda runs your code in response to events.
        (Events could be changes to data in an Amazon S3 bucket or an Amazon DynamoDB table).
      - Compute service to run your code in response to HTTP requests using Amazon API Gateway or API calls made using SDKs
      - Lambda can do things globally, you can use it to back up S3 buckets to other S3 buckets etc.
- - - - One record with multiple IP addresses
    - - Example: 20% go to THIS and 80% go to THAT
    - - Will send them to the place with lowest latency (the faster one to connect to)
    - - Primary side and passive side. If entire region go down Route 53 detect failure and redirect traffic to the passive place. Uses health checks
    - - Figures out the user location and redirects them to the correct place.
    - - We have healthcheck against every route. If one goes outOfService it redirects to the other one.
        Pretty much an updated version of Simple Routing