Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chaos Engineering in Healthcare systems (Chaos Experiments (Prepare (PLan…
Chaos Engineering in Healthcare systems
Our Story (Cerner)
Challenge
Lots of services
COmplex environment
Growing in size
Critical workloads
KUBERNETES
This was where they needed to move to
GOod moves
INstalled Spinnaker to help deploy to DC/OS (for OpenStack
and then Kubernetes
Tech Stack
Java
WebSphere
Host a lot of clients
RUn their own datacentre
Introduced OPenStack
Why
Clean AZ separation
No shared resources
Desires
Incrementally build software
INfrastructure as Code
Cross-team alignment
Techniques
INtroduced
TIgerTeam
CO-LOCATED PLATFOMRS, OPS AND INFRA
Game Days
Simulate traffic while killing VMs
Traffic Management Patterns
Shadow Traffic
Replays of Production traffic
Only safe traffic
e.g. HTTP GET
Tech
API Gateway
Used as a control gate into the system
Abstract traffic
ZUUL
Chaining Traffic
Supports API Gateway
Worked for static traffic
Canary Traffic
Gradual control of migrating traffic
Avoids
Big Bang
Chaos Experiments
Helped to transition to Kubernetes
Align INtroduction
With organised experiments
Avoid
Where it can become a scape goat for other problems
Prepare
Describe the scenario
What is expected to occur
How it will be measured
Who is needed
Dedicated space for all participants
PLan to be surprised
They will always happen
Observability
This is critical
Need easy access to telemetry data
Summary
remind your teams on the measurable improvements this work has achieved
Takeaways
They have a Platform Engineering Team