Please enable JavaScript.
Coggle requires JavaScript to display documents.
Cloud Foundry: The Definitive Guide --Duncan C. E. Winn (Concepts (…
Cloud Foundry: The Definitive Guide
--Duncan C. E. Winn
cf offer
Agile and automation
Cultural shift to DevOp
Containers
Microservices suppor
Services as a higher level of abstraction
above infrastructure
Cloud-native application support
Concepts
:pencil2:The Twelve-Factor Contract
:pencil2:Release Engineering through BOSH
:pencil2:Self-Service Application Life Cycle
:pencil2:Staging
:pencil2:Built-In Resilience
and Fault Tolerance
achieves resiliency and
self-healing through
Deployment of new AIs if an application crashes or becomes unresponsive
Application striping across availability zones (AZs) to enforce separation of the underlying infrastructure
Recreating missing or unresponsive VMs
Dynamic routing and load balancing
Restarting failed system processes
:pencil2:Using cf push
Command to Deploy
Uploads and stores application files
Examines and stores application metadata
Stages the application by using a buildpack to create a droplet
Selects an appropriate execution environment in which to run the droplet
Starts the AI and streams logs to the Loggregator
:pencil2:Aggregated Streaming of
Logs and Metrics
Loggregator: manage the performance, health, and
scale of running applications and the platform itself
Logs provide visibility into behavior
Metrics provide visibility into health
The Loggregator system architecture used for aggregating application logs and metrics Page65
The benefits of aggregated log, metric,
and event streaming
You can stream logs to a single endpoint.
Streamed logs provide timestamped outputs per application.
Both application logs and system-component logs are aggregated, simplifying their consumption.
Metrics are gathered and streamed from system components.
Operators can use metrics information to monitor an instance of Cloud Foundry.
You can view logs from the command line or drain them into a log management service such as an ELK stack (Elasticsearch, Logstash, and Kibana), Splunk, or PCF Metrics.
Viewing events is useful when debugging problems.
:pencil2:The Application as the Unit of Deployment
:pencil2:Security
Distributed System Security
How much effort is required to automatically establish
and apply network traffic rules to isolate components?
What policies should be applied to automatically limit resources in order to defend against denial-of-service (DoS) attacks?
How do you implement role-based access controls (RBAC)
with in-built auditing of system access and actions?
How do you know which components are potentially
affected by a specific vulnerability and require patching?
How do you safely patch the underlying OS without incurring
application downtime?
Environmental Risk Factors for Advanced
Persistent Threats(APT高级持续威胁)
need three risk factors
Leaked or misused credentials
Misconfigured and/or unpatched software
Time
Challenge of Minimal Change
The Three Rs of Enterprise Security:
rotate, repave, repair
Repave (rebuild) servers and applications from a known good state to cut down on the amount of time an attack can live
Repair vulnerable software as soon as updates are available
Rotate the credentials frequently so that they are
valid only for short periods of time
ADDITIONAL CLOUD FOUNDRY SECURITY
Manages software-release vulnerability by using new Cloud Foundry releases created with timely updates to address code issues
Manages OS vulnerability by using a new OS created with the latest security patches
Implements RBACs, applying and enforcing roles and permissions to ensure that users of the platform can view and affect only the resources to which they have been granted access
Secures both code and the configuration of an application within a multitenant environment
Deploys each application within its own self-contained and isolated containerized environment
Prevents possible DoS attacks through resource starvation
Provides an operator audit trail showing all operator actions applied to the platform
Provides a user audit trail recording all relevant API invocations of an application
Implements network traffic rules (security groups) to prevent system access to and from external networks, production services, and between internal components
:pencil2:The Cloud Operating System
Cloud Base OS
Bosh
Cloud Provider Interface
Cloud Foundry
Apolication Layer
Infrastructure Layer
:pencil2:Removing: Undifferentiated Heavy Lifting
Provisioning VMs, OSs, middleware, and databases
Application runtime configuration and memory tuning
User management and SSO integration
Load balancing and traffic routing
Centralized log aggregation
Scaling
Security auditing
Providing fault tolerance and resilience
Service discovery
Application placement and container creation and orchestration
Blue/green deployments with the use of canaries
:pencil2:UAA Management
:pencil2:Organizations and Spaces
Provide:
Logical separation and assignment of
Cloud Foundry resources
solation between different teams
Logical isolation of development, test, staging,
and production environments
Resource Allocation
Resource quota
Applications
Services availability
Custom domains
:pencil2:Domains Hosts and Routes
:pencil2:Rolling Upgrades and Blue/Green Deployments
Components
Application Execution
Diego
Garden and runC
Metrics and Logging
Metron Agent
Loggregator
cf logs APP
The Cloud Controller
interact cf three way
A scriptable CLI
Language bindings (currently Java)
Integration with development tools (IDEs) to
ease the deployment process
System State:
storing state
The CF blobstore
Application code packages—unstaged files
that represent an application
Resource files
Buildpacks
Droplets and other container images
The CF CCDB
maintains records of the logical hierarchical structure including available orgs, spaces, apps, services, service instances, user roles, and more
maintains users’ information and how their roles map to orgs and spaces
exposes Cloud Foundry’s REST API
Pushing, staging, running, updating, and retrieving applications
Pushing, staging, and running discrete one-off tasks
Messaging
A Consul server stores the longer-lived control data such as component IP addresses and distributed locks that prevent components from duplicating actions”
Diego’s bulletin board system (BBS) stores a real-time view of the system, including the more frequently updated and disposable data such as Cell and application status, unallocated work, and heartbeat messages
User Management and the UAA
own database:UAADB
Cloud Foundry developers
Application clients/end users
Applications requiring application-to-application interactions
Additional Components
Infrastructure and the Cloud Provider Interface
Storage for VMs
File server or blobstore
Ms with specified CPU and memory requirements
DNS, certificates, and wildcard domains
Networks and subnets
Load balancer to pass traffic into the GoRouter
NAT for traffic flowing back to the load balancer
Buildpacks and
Docker Images
cf push:
two defined artifact types
A standalone application
A prebuilt Docker image
The buildpack
automates the following
The detection of an application framework
The application compilation
Running the application
A Marketplace of
On-Demand Services
Service brokers
List service offerings
Provision (create) and deprovision (delete) service instances
Enable applications to bind to, and unbind from, the service instances
User-provided services
expose existing services via user-provided services.
Stacks
A stack is a prebuilt root filesystem (rootfs)
Stacks are used along with droplets
(the output of buildpack staging)
Routing via the Load Balancer and GoRouter
Component Overview
Cloud Foundry component layers(Page95)
Routing:
GoRouter, TCPRouter, and external load balancera
Authentication and user management:
User Access and Authentication Management
Application life cycle and system state:
Cloud Controller, Diego’s core components (e.g., BBS and Brain)
App storage and execution:
blobstore (including app artifacts/droplets and the Application Life-Cycle Binaries), Diego Cell (Garden, and runC)
Services:
Service Broker, User Provided Service
Messaging:
NATS (Network Address Translation) Messaging Bus
Metrics and logging:
Loggregator (including Doppler and the Firehose)
Preparing Your Cloud Foundry
Environment
Deployment Topology
部署拓扑 :red_flag:
......
page127
How many Cloud Foundry instances should you deploy?
There are a number of factors to consider when
addressing this question
Do you need one instance for the entire company or one instance per organization?
Should you have one instance for preproduction workloads and a separate instance for production apps?
Do you need to isolate applications or data due to regulatory reasons such as PCI, NIST, or HIPAA compliance?
Do you need high availability of Cloud Foundry itself (data center-level redundancy)?
Non-technical
Considerations :red_flag:
two critical things
The team structure required for installing CF
The required deployment topology of CF
Team Structure:
Platform Operations for the Enterprise
Networking administrator
Storage administrator
System administrator
IaaS administrator
Software development
Security
QA and performance testing
Release management
Project management
Installation Steps :red_flag:
Create and configure your IaaS environment, including all the periphery infrastructure that Cloud Foundry requires, such as networks, security groups, blobstores, and load balancers.
Set up any additional external enterprise services such as LDAP, syslog endpoints or monitoring, and metrics dashboards.
Deploy the BOSH Director.
Create an IaaS/infrastructure-specific BOSH configuration such as cloud configuration.
Create a deployment manifest to deploy Cloud Foundry.
Integrate Cloud Foundry with the required enterprise services (via your deployment manifest).
Deploy Cloud Foundry.
CF Dependencies and
Integrations :red_flag:
the minimum external dependencies:
Configured networking (subnets and security groups)
Defined storage policy and an additional NFS or Amazon S3–compatible blobstore (for both the BOSH blobstore and CF blobstore
Configured IaaS and infrastructure environment with available administrator credentials
External load balancers set up to point to GoRouter IP addresses
DNS records set up, including defining appropriate system, app, and any other required wildcard domains along with SSL certificates
Additional integration considerations based on
enterprise services may require the following:
A syslog endpoint such as Splunk or ELK (Elasticsearch, Logstash, and Kibana) is available to receive component syslog information.
SAML, LDAP, or SSO configured for use with Cloud Foundry where required
System monitoring and metrics dashboards such as DataDog set up to receive system metrics.
An application performance management (APM) tool such as Dynatrace, NewRelic, or AppDynamics set up for receiving application metrics
IaaS and Infrastructure
Design :red_flag:
BOSH CPIs
Google Compute Platform
AWS
Azure
OpenStack
vSphere’s vCenter
vSphere’s vCloudAir
Photon
RackHD
Your local machine (for deploying BOSH Lite)
Designing for Resilience
use of AZs to drive antiaffinity of components and applications to physical servers.
Cloud Foundry AZs
Sizing and Scoping the Infrastructure
Require a minimum of seven instances when running in an active–active setting
Page139
Cell sizing
a single application might require
several instances
Running multiple instances for resilience
Running multiple instances over different spaces
throughout the application life cycle
Running multiple instances for concurrency performance
Setting Up an AWS VPC
dependencies
Access key ID
AWS secret key
VPC ID
AWS security group name
Region
Key-pair name (used for VM-to-VM communication and SSH)
DNS IP(s)
NTP IP(s)
SSH password (if applicable)
VPC to data center connectio
Jumpbox
Networking Design and Routing :red_flag:
Network dependencies
AWS network name
VPC subnet ID
Subnet (CIDR range)
Excluded IP ranges
DNS IP(s)
Gateway
Using Static IPs
Load balancer (static IP for HAProxy, VIP for F5, etc.)
GoRouter (depending on the IaaS, some CPIs will resolve this for you)
NATS
Consul/etcd
Database such as a MySQL cluster (relational databse service [RDS] referenced by DNS)
Subnets
a common practice for
production environments
A management subnet for
the BOSH Director and or jumpbox
A dedicated subnet for the core Cloud Foundry components (which also could contain the BOSH Director)
A dedicated subnet for the Diego Cells (so as to scale independently as apps scale)
A dedicated internet protocol security (IPsec) subnet for services that require the use of IPSec
A dedicated subnet for services that do not use IPSec
Security Groups
Setting Up the Load Balancer
Setting Up Domains and Certificates
need a registered wildcard domain for its default domain
need this registered domain when configuring your
SSL certificate and Cloud Controller.
recommended at least
two default wildcard
*.system.cf.com
*.apps.cf.com
It is also recommended
*.system.apps.cf.com
*.apps.cf.com
Installing and Configuring CF
Installation Steps
Deploy the BOSH Director.
Create an IaaS/infrastructure-specific BOSH
configuration such as cloud configuration.
Create a deployment manifest to deploy CF
Integrate CF with the required enterprise
services (via the deployment manifest).
Deploy Cloud Foundry
Installing CF
BOSH environment consists of both the Director and the deployments that it orchestrates( 编排)
$bosh create-env provides only the Director
install both BOSH and the required IaaS
environment using bosh-bootloader
Before using
bosh-bootloader:
$ wget
https://github.com/cloudfoundry/bosh-bootloader/releases/download/v2.3.0/bbl-v2.3.0_osx
$ chmod +x bbl-v2.3.0_osx
$ mv bbl-v2.3.0_osx /usr/local/bin/bbl
Add the correct inline policy to your AWS user
$ mkdir bosh-bootloader;cd bosh-bootloader
Export any required environment variables:
$ export BBL_AWS_ACCESS_KEY_ID=<>
$ export BBL_AWS_SECRET_ACCESS_KEY=<>
$ export BBL_AWS_REGION=<>
BBL_AWS?
create-env provides only the Director
Focus on bosh-bootloader
To set up the AWS VPC and deploy the BOSH Director
$ bbl up
Pull out the Director IP, ca-cert, username, and password from the bbl CLI and then log in to your BOSH Director.
$bosh alias-env my-bosh -e <YOUR-BOSH-IP>
$bosh -e my-bosh --ca-cert <(bbl director-ca-cert) login --user $(bbl director-username) --password $(bbl director-password)
Create the ELB using the key and certificate you created
$ bbl create-lbs --type cf --cert cert.pem -key key.pem
Upload the appropriate stemcell for your Iaas
bosh -e my-bosh upload-stemcell
https://bosh.io/d/stemcells/
...
Deploy Cloud Foundry.
bosh -e my-env -d cf deploy cf-deployment/cf-deployment.yml --vars-store env-repo/deployment-vars.yml -v system_domain=<YOUR-CFDomain.com>
you should use config-server instead of var-store.
Target the cf api domain and then log in
cf api api.<YOUR-CFDomain.com> --skip-ssl-validation
Changing Stacks
$ cf stacks
$ cf push APPNAME -s STACKNAME
Diego
Interacting with Diego
CAPI
API interaction from the platform user through to Diego:
User--->CAPI(cloud controller)--->Translate Bridge(
CC-Bridge)--->Diego Api(BBS)
Cloud Controller provides
REST API: Commands
Pushing, staging, running, and updating
Pushing and running discrete one-off tasks
Creating users, Orgs, Spaces, Routes,
Domains and Services, and so on
Retrieving application logs
Staging Workflow
Two Diego components
The BBS: Diego’s database that exposes the Diego API
Cells: the execution machines responsible for running applications in containers.
Figure 6-4. Interaction between Cloud Foundry’s Cloud Controller components and Diego, while staging and running an application
Page 231
The CC-Bridge
The CC-Bridge is a special component
comprised of four microservices.
cc-uploader:
A file server to serve static assets to the
Cloud Controller’s blobstore.
nsync:
This service is responsible for handling domain
freshness from the Cloud Controller to Diego.
Stager:
The Stager handles staging requests.
TPS:
This service is responsible for handling domain
freshness from Diego to the Cloud Controller
The components on the CC-Bridge are
essential for establishing domain freshness
The actual state reflects the desired state
The desired state is always understood
Logging and Traffic Routing
Diego, Loggregator, and
the routing subsystem
Route traffic to the LRPs
Stream logs from the LRPs
Run any number of applications as a single user
CF users do not
interact with Diego directly
user-facing components that
work in conjunction with Diego
The logging system defined by the
Loggregator and Metron agents
Routing (GoRouter, TCPRouter,
and the Route-Emitter)
CAPI components:
(Cloud Controller and the CC-Bridge)
these CF components are
responsible for the following
Application policy
Uploading application artifacts, droplets,
and metadata to a blobstore
Traffic routing and handling application traffic
Logging
User management including end-user interaction
via the Cloud Foundry API commands
Layered Architecture
Diego is comprised of a number of microservices
residing on several components.
Figure 6-2. Diego components: page223
Essential Diego Concepts
two fundamental Diego concepts
Action abstraction
Composable actions
COMPONENTS VERSUS SERVICES
In BOSH parlance, components are referred to as instance groups and services are referred to as release jobs.
Composable actions include
the following
Composable actions include the following:
RunAction runs a process in the container.
DownloadAction fetches an archive (.tgz or .zip)
and extracts it into the container.
UploadAction uploads a single file, in the
container, to a URL via POST.
ParallelAction runs multiple actions in parallel.
CodependentAction runs multiple actions in parallel
and will terminate all codependent actions after any
single action exits.
SerialAction runs multiple actions in order.
EmitProgressAction wraps another action and logs messages
when the wrapped action begins and ends.
TimeoutAction fails if the wrapped action
does not exit within a time interval.
TryAction runs the wrapped action but ignores
errors generated by the wrapped action.
Applications are broken into a set of Tasks such as “stage an app” and “run an app.” All Diego Tasks will finally result in a tree of composable actions to be run within a container
Why Diego?
Two types of processes：
Task
Long-running process (LRP)
Diego handles the scheduling, running, and monitoring of tasks and long-running processes (applications)
supports the original droplet plus a stack combination
accommodate other image formats:
OCI-compatible images such as Docker or Rocket
Garden-based container technology, including Linux、Windows-based container backends that implement the Garden API
Diego Components
The BBS