Please enable JavaScript.

Coggle requires JavaScript to display documents.

Google Cloud Platform (GCP Introduction (GCP Overview (Services…

- - - - Challenge: Need Concise and Accurate Demand from IT in Advance
      - Issue: If you fail...either you procure too much H/W and S/W and spend much $$$ or too less and you fail to because of high load
      - Benefits of Cloud Computing
        
        Ease of use
        
        Provision IT resources in Mins
        
        Provides Elasticity
        
        Pay as you GO service - Opex and not Capex
        
        Implement Security the way you wanted
        
        All of the above contributes to:
        
        Quick Turn Around time to your business
        
        Quick Implement any business without restricting infrastructure limitations - no IT bottlenecks
        
        No precise / Exact planning required for IT demand
        
        Resources are practically unlimited in terms of H/W and S/W available and provisioned for you
        
        Additional Benefits
        
        No need to manage H/W and S/W
        
        Many ready made integrations with third party software's
        
        Go from Zero to Planet-Scale in few mins
        
        Focus on your Business and not IT
        
        Micro Services Architecture... Scaling Limitless
        
        Don't need to monitor your demand, its automatic scalable - using Auto Scaling
        
        Have multiple Software platforms ready to use as SAAS as Integrated envs - Analytics, Machine Learning, BI, CRM etc
    - - Google Cloud Platform
      - Amazon Web Services
      - Microsoft Azure
    - - 60% questions asked on 4 Case Studies, 40% Scenario based, 20% of the 60% would be direct service questions - example. how to increase performance in a compute service in this particular scenario, or what options from the different database or storage service will you use for a particular situation
      - $200, Length: 2hrs, 50 questions multiple choice, multiple select
      - Need to know each and every service, their performance characteristics, their provisioning characterstics
      - Core Services: Compute, Storage & Database, Networking, Auto-scaling / HA
        Supporting Services: IAM & Security, Management - Logging and Monitoring, Developer and Management Tools
        Other Services: Big Data, Machine Learning, IOT
      - Case Studies
        
        Case Study 1: MountKirk Games
        
        Profile: Online Gaming Platform for Mobile Users or Desktop
        
        Current Architecture: Server Side Integration, All data loaded in MySQL DB for reporting and running analytics
        
        Problem Statement: Unable to Scale Application Servers (backend), MySQL databases and Analytics tools
        
        Requirement: ToBE: Building new Game, Plan to deploy Games back-end on Google Compute Engine. Requirement: Capture streaming metrics, run intensive analytics and take advantage of its auto-scaling server enviornment and integrate with a managed NoSQL database. Need KPI's to evaluate the speed and stability of the game. Need to provide deeper insights in to usage patterns
        
        Historical Challenges & Issues: Unable to scale well, drop in user adoption affecting the game reputation. Current technology stack cannot provide the scale we need, so we want to replace MySQL and move to an enviornment that provides autoscaling, low latency load balancing and frees us from managing physical servers. Not capturing enough user demographic data, usage metrics and other KPIs
        
        Clearly Specified Requirement:
        
        Business Requirements
        
        Increase to a global footprint
        
        Improve uptime - downtime is loss of players
        
        Increase efficiency of the cloud resources we use
        
        Reduce latency to all customers
        
        Technical Requirements
        
        Requirements for Game Backend Platform
        
        5 more items...
        
        Requirements for Game Analytics Platform
        
        5 more items...
        
        Case Study 2: JencoMart
        
        Case Study 3: TerramEarth
        
        Case Study 4: Dress4Win
  - - - Renewable Energy
      - Efficiency
      - Data & Security
      - Own Servers
    - - Has Multiple locations and are building many more
    - - Edge or POP locations across the globe
      - POP locations supports CDN (Content Delivery Network) - providing caching abilities
    - - Located Across: NA, Europe, Asia, Australia,
        
        Locations are divided in to Regions and Zones
        
        Regions are independent
        geographic area that consists of zones
        
        Locations within the region tend to have route trip network latency under 5 ms i.e. 95%tile
        
        Zone is a deployment area for cloud platform resources within the region
        
        Zone should be considered as a single failure domain within the region
        
        High Availability applications should be deployed across multiple zones within the region to help protect against unexpected failure
    - - Global
        
        Resources can be accessed across the globe
        
        Includes pre-configured Disk Image, Disk Snapshot & network
      - Regional
        
        Resources can be accessed by the resources located in the same region
        
        Static IP Addresses
      - Zonal
        
        Other resources can only be accessed by the resources located in the same zone. These are zonal resources
        
        Virtual Machine Instances, Their types and disks
    - - Any cloud platform resources that you allocate and you use must belong to a project - an organizing entity for what you are building
      - Project is made up of Settings, Permissions and other metadata that describe your application
      - Resources within a single project can work together easily
      - Has a project name provided by you, project id - provided by you or gcp will provide, and the project number which is unique # provided by cloud platform
      - Interactions:
        GCP Console - Web based GUI used to manage GCP Projects and resources.
        Command Line Interface: gcloud commands using google cloud sdk
        Client Libraries: Cloud SDK includes client libraries that enable you to easily create and manage resources - APP APIs & Admin APIs
    - - Compute Service
        
        Compute Engine (GCE) (IAAS)
        
        High Control on Infra
        
        Actual Infrastructure layer of GCP
        
        Scalable, High Performance Virtual Machines
        
        Runs on Googles innovative data centers and worldwide fiber network
        
        Use Cases - When to choose Compute Engine:
        
        Requirement can't be easily containerized using existing VM images
        
        Application that needs OS level changes
        
        Wan't to host application without rewriting it
        
        Or you have a requirement, where you want control over infrastructure and direct access to Network/Other Compute resources
        
        Product Features:
        
        VMs with n/w attached and ultra high performance local storage options (SSD's)
        
        Cost Effective Preemptible VMs for inexpensive batch jobs and fault-tolerant workloads
        
        Customizable load-balancing and auto-scaling across homogeneous VMs
        
        Direct access to GPU's that you can use to accelerate specific workloads
        
        Support for the most popular flavours of Linux and Windows OS (User Managed rest all Google Managed)
        
        Comes with Predefined and Custom Machine Types
        
        n1 - standard machine
        f1 - shared core machine
        G1 - shared core machine
        High CPU Machine
        High Memory machine
        Custom Virtual Machine
        
        Default:
        
        1 10 GB Persistent Disk
        
        Longer RAM: 2 GB of RAM per core - CusVM - Custom Virtual Machine: 3.75 GB RAM each core - so 2 cores would be 7.5 GB of RAM
        
        Machine type - n1, f1, g1 : Default CusVM
        
        4 Cores
        
        Predefined
        
        Standard Class (Used for 90% of your workload - suitable for balance requirements of your CPU and Memory needs. They have 3.75 GB of RAM per Virtual CPU . Max no of PDs - 16. Max total PD Size in TB - 64)
        
        High Memory (Used when more memory is required to cache the data than than the processing power. They have 6.50 GB of RAM per Virtual CPU . Max no of PDs - 16. Max total PD Size in TB - 64)
        
        High CPU (Used when more processing power is required than caching or memory. They have 0.90 GB of RAM per Virtual CPU . Max no of PDs - 16. Max total PD Size in TB - 64)
        
        Shared Core (Used when you don't need a full CPU)
        
        f1 micro - Micro machine type with 0.2 virtual CPU, 0.60 GB of memory, backed by a shared physical core. Max PDs - 4. Max total PD Size in TB - 3
        f1 allows micro bursting - Means, Instance allows additional CPU usage for short period of time. Bursting happens automatically.
        
        g1 small - Share core machine type with 0.5 virtual CPU, 1.70 GB of memory, backed by a shared physical core. Max PDs - 4. Max total PD Size in TB - 3
        
        Custom Machine Type (Slightly More expensive than predefined)
        
        Ideal for the following scenarios:
        
        Workloads that are not a good fit for the predefined machine types
        
        Workloads requiring more processing power or more memory, but don't need all of the upgrades that are provided by the next larger predefined machine type
        
        Limitations
        
        Only machine types with 1 vcpu or even number of vCPU's can be created
        
        Memory must be between 0.9 GB per VCPU, up to 6.5 GB per VCPU
        
        Total memory of the instance must be a multiple of 256 MB
        
        Max # of vCPUs allowed depends on the zone
        
        Supports Global Load Balancing
        
        GCP offers server-side internal load balancing so you can distribute incoming traffic across multiple VMs
        
        Its a Managed Component, so redundant and highly available
        
        Benefits:
        
        Scale your application
        
        Supports heavy traffic
        
        Detection and Auto removal of Unhealthy VM instances
        
        Route traffic to closest VMs
        
        Supports Auto Scaling
        
        GCE offers autoscaling to automatically add or remove VMs from an instance group based on increases or decreases in load
        
        Gracefully handle increase in traffic and reduces cost when the need of resources is lower
        
        Define Auto Scaling Policy and the autoscaler performs automatic scaling based on the measured load
        
        CPU Utilization Policy
        
        Load Balancing Serving Capacity Policy
        
        Stackdriver monitoring metrics Policy
        
        Google Cloud Pub/Sub queuing workload (in Alpha stage right now) Policy
        
        Provides Sub Minute Billing
        
        Globally Scoped disk images
        
        Integrated Networking
        
        High Network Performance
        
        Live Migration - Transparent Maintenance
        
        Automatic Discount
        
        Min level increment beyond 1 min
        
        Committed Savings
        
        GCE Resources used when creating a VM
        
        Disks
        
        Images (Contains - a boot loader,
        an OS, and a root file System)
        
        Private Images - Maintained by you
        
        Import a boot disk image to CE
        
        Create an Image from root persistent disk
        
        Copy one image to another image
        
        Public Images - Maintained by Google
        
        Community Supported Images
        
        Networks
        
        Firewalls
        
        Addresses
        
        Regions and Zones
        
        Instances (User Managed rest all Google Managed)
        
        Load Balancers
        
        Global
        
        Global External Load Balancing
        
        3 more items...
        
        Regional
        
        Regional external load balancing
        
        1 more item...
        
        Regional Internal load balancing
        
        1 more item...
        
        Instance Templates (defines the machine types, image, zone, and other instance properties for the instance in a managed instance group)
        
        Create once and re-use
        
        Global resource, not bound to zone or a region
        
        Instance Groups (are group of VM instances so that you don't have to individually control each instance in your project)
        
        Managed Instance Groups -
        Used for Auto Scaling
        Use identical resources
        Rolling updates can be done
        Change to instance will make changes to all instances
        Load balancing for only similar resources
        
        Zonal Managed
        
        Regional Managed
        
        Unmanaged Instance Groups -
        Cannot be used for Auto Scaling
        Use non-identical resources
        Rolling updates cannot be done
        Can make arbitrary changes in any instances
        Can be used for load balancing pre-existing resources or Groups of dissimilar resources
        
        GCE Instance Lifecycle:
        
        Provisioning - instance is not running yet but resources committed
        
        Staging - Resources acquired and instance is prepared for launch
        
        Running - Instance is booting up and running. You should be able to SSH in to the instance, though not immediately
        
        Stopping - Instance is being stopped - either due to failure or instance is being shutdown. This is a temporary status and eventually the status will move to Terminated
        
        Terminated - Instance was shutdown or encountered a failure, either through the API or from inside the guest. You can choose to restart the instance or delete it
        
        App Engine (GAE) (PAAS)
        
        Serverless (Don't have much control on the resources)
        
        Used for Web Apps and APIs
        
        A flexible, zero ops platform
        
        Platform for building scalable web applications, mobile and IOT backends
        
        Provides builtin services & APIs, such as NOSQL datastore, memcache, and a user authentication API, common to most applications
        
        Use Cases - When to choose App Engine:
        
        Focus on writing code, and never want to touch a server, cluster, or infrastructure
        
        Developer Velocity over Infrastructure control
        
        Minimize operational overhead
        
        Build reliable and scalable serving app or component without doing it all yourself
        
        Product Features:
        
        Fully Managed
        
        A stack with smart defaults and deep customizability
        
        Support for Java, Python, PHP, Go, Ruby(beta), and Node.js (beta)... or bring your own app runtime
        
        integrated SDK, managed services, and local development enviorhment
        
        App versioning with zero-downtime upgrades
        
        Traffic Splitting
        
        Automatic High Availability with built-in autoscaling
        
        How to Deploy your app- 3 step process:
        
        gcloud init
        
        gcloud auth login
        
        gcloud app deploy
        
        2 types of Instance scaling
        
        Manual Scaling - will run exact same number of instances irrespective of the load. Used for tasks such as complex initialization, or applications that rely on the state of memory over a period of time
        
        Automatic Scaling - Based on the request, response based on other metrices
        
        Machine Types (Engine Type)
        
        Standard Environment
        
        Runtime - Predfined runtime containers
        
        Infrastructure Customizations - You can't customize env. it has to be selected from preconfigured env
        
        Performance - Applications run in a secure, sandboxed enviornment, allowing app engine standard enviornment to distribute requests across multiple servers, and scaling servers to meet traffic demands. Your application runs with its own secure, reliable enviornment that is independent of the hardware, OS, or physical location of the server
        
        Flexible Environment ( Is dockerized container image, you can build your own and deploy where standard environment is with pre-defined run time which is available for you. You have SDK, you can use it and deploy your application)
        
        Runtime - Customized Runtime, Custom Docker image or docker file
        
        Infrastructure customizations - you have GCE VM's underlying, so you can take advantage of custom libraries, use SSH for debugging, and deploy your own docker containers
        
        Performance - You can specify which and how much CPU and Memory each instance of you application needs the flexible env will provision the infra for you
        
        Container Engine (GKE, Kubernetes) (PAAS, Containers)
        
        High Control on Infra
        
        Orchestrate Containers
        
        Deploy, Manage & Scale Containerized Applications
        
        Powerful Cluster Manager and Orchestration System for running your Docker Containers
        
        Use Cases - When to Choose Container Engine:
        
        Applications should be containerized which means you want to separate the app from the OS
        
        Don't have dependencies on a specific operating system
        
        You wan't to have a secure, salable way to manage containers in production
        
        Product Features:
        
        Auto Scaling and High Available Cluster/Container management and Orchestration
        
        Seamless computing resource provisioning across the globe
        
        Logical infrastructure - focus on your app components and not on VMs
        
        Easy mechanisms for building loosely-coupled distributed systems
        
        Run the same application on you laptop, on premise and in the cloud - other container advantages
        
        Container Engine Cluster is a group of compute engine instances running kubernetes
        
        Consists of one or more node instances, and a managed kuberenetes master endpoint
        
        Container Cluster is the foundation of a Container Engine application - pods, services, and replication controllers all run on top of a cluster
        
        Auto Upgrade, Auto Repair, Hybrid Networking
        
        Integrated Logging & Monitoring
        
        Stateful Application
        
        Docker Image Support
        
        Private Container Registry
        
        Identity and Access Management
        
        Container Registry
        
        Cloud Functions (PAAS)
        
        Serverless (Don't have much control on the resources)
        
        Used for Event Driven Functions
        
        Serverless enviornment to build and connect cloud services
        
        Fully Serverless models where logic can spun-up on-demand in response to events originating from anywhere
        
        Construct applications from bite-sized business logic billed to the nearest 100 milliseconds, only while your code is running
        
        Use Cases - When to Choose Cloud Functions
        
        Event based Code-Business Logic execution
        
        Absolutely no operations overhead
        
        Do not wan't to run app when no one is using it
        
        Simple to implement with multiple end points
        
        Product Features:
        
        True Serverless with Serverless Economics (Cloud Functions are Empemeral, spun up on demand. Pay only while your function is executing)
        
        Connect and Extend Cloud Services
        
        Mobile Ready Platform
        
        Just Add code
        
        Open Source Support
        
        Best Use Case - IOT
        
        Use to build API and Microservices
        
        Can trigger Data Processing / ETL jobs
      - Networking Services
        
        Cloud Virtual Network
        
        VPC Network (Managed Networking Functionality)
        
        Cloud Load Balancing
        
        Autoscaling & Load Balancing (High Performance, scalable load balancing)
        
        Cloud CDN
        
        Low Latency, Low Cost Content Delivery
        
        Cloud Interconnect
        
        Connect Infrastructure at Google's network edge with enterprise-grade interconnect
        
        Cloud DNS
        
        Reliable, Resilient, Low Latency DNS Serving
      - Storage & Database Services
        
        Cloud Spanner (PAAS for Database) - Fully Managed Mission Critical Relational DB service. Map Spanner to any other RDBMS. Google Proprietary not any commercial RDBMS like MySQL or PosgresSQL
        
        When to Choose:
        
        Mission Critical applications. High Transactions
        
        Scale + Consistency requirements
        
        Adtech, Financial, Global supply chain, Retail
        
        All RDBS requirement
        
        Horizontally Scalable ( Scale to even 1000s of Servers)
        
        Cloud Storage (PAAS) - Objects Storage - Images, Videos, Files - Unstructured Data
        
        When to Choose:
        
        Images, Pictures, Videos or Objects, Blobs or Unstructured Data
        
        Storing and Streaming Multimedia
        
        Archive, backup , and disaster recovery
        
        Storage for custom data analytics pipelines
        
        Designed for 99.9999999% durability. Stores data redundantly and maintains data integrity. Multi-regional storage - data maintained in distinct geo locations
        
        Availability: Multi-regional - 99.95% and regional storage offers - 99% monthly availability
        Nearline and ColdLine storage offer 99% monthly availability
        
        Scalable - Practically infinitely scalable
        
        Consistency - When write succeeds, latest copy of the object is guaranteed to be returned to any GET, globally. Applies to PUTs of new or overwritten objects and DELETEs as well
        
        Use Cases
        
        Server Rich Content - Multi Regional Storage class
        
        Analyze existing data
        
        Archive Cold data - Nearline/Coldline
        
        Build your mobile apps
        
        Host your website
        
        Make your data work for you
        
        Upload your data
        
        Storage Class
        
        Multi-Regional - frequently accessed data. Hot object. Streaming Videos & Music. Serving images and website content. Optimized for end user latency (serves from closest available location)
        
        Regional - redundant within the region. Lower cost than Multiregional. Appropriate for storing data used by GCE instances, Used for - Genomic Analysis and Data Transcoding, Analytics for ecommerce and IOT, Integrated with Cloud Dataproc, CloudML, BigQuery. Storage connector is HDFS compliant. Solved Problems with - 1) High Availability, Scale and Management Overhead 2)Persists outside of clusters and available for other use
        
        Nearline - low cost storage for storing infrequently accessed data. Slightly Lower availability. 30-day minimum storage duration. Ideal for data accessed on an average once a month or less. Also good for DR, Data Backup and Archival storage
        
        Coldline - very low cost, highly durable storage for data archiving, online backup and DR. Data is available within milliseconds, not hours or days. Best for data accessed atmost once a year, due to its slightly lower availability, 90-day minimum storage duration, costs for data access, and higher per-operation costs
        Data stored for legal or regulatory reasons. Disaster Recovery data.
        
        Cloud Bigtable (PAAS for Database) - Highly Scalable - NOSQL db service - Suitable for low latency, high trhoughput. Map BigTable to Hbase. Columnar DB.
        
        When to Choose:
        
        Low latency read/write access. High Throughput analytics
        
        Native timeseries support. IOT, finance, adtech
        
        Personalization, recommendations, Monitoring
        
        Geospatial datasets, Graphs
        
        Scalable, NO SQL
        
        Fast and Performant Low Latency
        
        Scaling and Resizing
        
        Simple & Integrated (Hbase, Hadoop)
        
        Global Availability, Fully Managed, Security & Permissions
        
        Cloud Datastore (PAAS for Database) - - Highly Elastic Document Oriented db service- Map to Mongo DB. NoSQL Document DB
        
        When to Choose:
        
        Semi Structured application data
        
        Hierarchical data durable key- value data
        
        User Profiles, Product Catalogs, Game State
        
        Document Data
        
        Scalable, NOSQL Document DB
        
        Multiple Access Method, Fully Managed, Diverse Data Types, ACID Transactions
        
        Cloud SQL (PAAS for Database) - RDBMS - Fully Managed RDBMS Service - MySQL & PostgresSQL
        
        When to Choose:
        
        Web frameworks Structured Data OLTP workloads
        
        Websites, blogs, and CMS, BI Applications
        
        ERP, CRM, and eCommerce applications, Geospatial applications
        
        All RDBS Requirement based on MySQL and PostgresSQL
        
        Simple and Fully Managed
        
        99.95% availability
        
        Performance and Scalability - upto 10 TBs and 25,000 IOPS - up to 208 GB of RAM, upto 32 processors
        
        Reliability and Security
        
        Persistent Disk (PAAS) - Disk for VMs - Can't use it idependently, attached to VMs - Fully managed Block Storage
        
        When to Choose:
        
        Block Storage for Google Compute Engine and Google Container Engine
        
        Rapid, durable backups of running virtual machines
        
        Sharing read-only data across multiple virtual machines
        
        Snapshots for data backup
        
        Automatic Encryption
        
        Industry Leading Price and Performance
        
        Durable, Volume Size, Independent Volume, Online Resize
        
        Storage Options for VMs
        
        PD for VM - Network attached storage (NAS)
        
        Local SSD - Attached to Physical Layer
        
        Google Drive - Free Tier - 15 GB of data storage
        
        Cloud Storage for Firebase - firebase is used for mobile framework.
        
        Firebase Realtime DB - Fully Managed Analytics DWH. Data Stored as JSON and synchronized in realtime. Used for Mobile Apps.
        
        BigQuery - is a Big Data Solution - Fully Managed Analytics DWH
      - Identity & Security
        
        Cloud IAM
        
        Cloud Resource Manager
        
        Cloud Security Scanner
        
        Cloud Platform Security
      - Big Data
        
        Big Query
        
        Cloud Dataflow
        
        Cloud Dataproc
        
        Cloud Datalab
        
        Cloud Pub/Sub
        
        Genomics
      - Machine Learning
        
        Cloud Machine Learning
        
        Vision API
        
        Speech API
        
        Natural Language API
        
        Translation API
        
        Jobs API
      - Management Tools
        
        Stackdriver
        
        Monitoring
        
        Logging
        
        Error Reporting
        
        Trace
        
        Debugger
        
        Deployment Manager
        
        Cloud Endpoints
        
        Cloud Console
        
        Integrated Cloud Platform Management Console for GCP
        
        Provides a Web Admin UI
        
        Perform Resource Management tasks
        
        DevOps on Go
        
        Secure Administrative Interface
        
        Get Data Insight
        
        Empowers Developer Productivity
        
        Cloud Shell
        
        Is your Admin machine in the Cloud - You
        can manage your infrastructure and applications from the command line in any browser
        
        Has a Admin Machine
        
        Pre-installed and up-to-date
        
        Full Power Access from anywhere
        
        Developer Ready
        
        Secure and Fully Authenticated
        
        5 GB Persistent Disk
        
        Limitations:
        
        Custom Installed Software packages and persistence
        
        Slow Connection performance
        
        Browser Support
        
        Usage Limits
        
        Outgoing connections
        only allowed ports: 20, 21, 22, 80,443, 2375, 2376, 3306, 8080, 9600 & 50051
        
        Cloud Mobile APP
        
        Manage GCP services from your Android or iOS device
        
        Cloud in Mobile App
        
        Production Alerting
        
        Diagnose and Fix
        
        Keep an eye on envs
        
        Billing App
      - Developer Tools
        
        Cloud SDK (Contains - gcloud tool,
        gsutil tool, Powershell cmdlets(Windows),
        bq tool, kubectl tool)
        
        Tools for Cloud Platforms
        
        Manage Virtual Machine
        
        N/w, Storage, Firewall, Disk and more
        
        Use your own favorite language
        
        Run Local Service Emulator
        
        Deployment Manager
        
        Cloud Source Repositories
        
        Multiple private Git repositories hosted on GCP
        
        Collaborative Development on Git
        
        Integrated with Cloud Diagnostic
        
        Connect Github or Bitbucket
        
        Source Code Browser
        
        Cloud Tools for Android Studio
        
        Cloud Tools for IntelliJ
        
        Cloud Tools for PowerShell
        
        Cloud Tools for Visual Studio
        
        Google Plug-in for Eclipse
        
        Mavin
        
        Graddle
        
        Cloud Test Lab
        
        Firebase Test Lab for Android
      - API & Eco-systems Management
        
        Google acquired apigee
        
        API Management
        
        API Monetization
        
        API Analytics
        
        APIGEE Sense
        
        Cloud Endpoints
      - Data Transfer
        
        Cloud Transfer Appliances
        
        Cloud Storage Transfer Service
        
        Cloud BigQuery Data Transfer Service