Please enable JavaScript.
Coggle requires JavaScript to display documents.
Processing Quality ((4) Optimise (App Refactor (Spark 2.0 (Datasets),…
Processing Quality
(4) Optimise
App Refactor
Spark 2.0
Datasets
Loops in Apps
Serialisation
Kryo
Garbage Collection
GC1
Infrastructure
Dig Big Data Recommentations
AWS
Machine types
https://www.packet.net/media/images/r6uQ-stress.testinglow.pdf
Architecture Change
More streaming
Composition of apps
rollup seperate
Consultancy
Dig Big Data
are our benchmarks ok?
perf test envs
(1) Stability
Spark Upgrade
App Tuning
Memory Allocation
Cores
Cluster Memory
64GB
Cassandra Upgrade
fault tolerance
greater than 3 nodes
(2) Visibility
Prometheus
Batch Apps
Spark/C* Connector
Workers
Executors
Push Gateway
Spark UI History
SRE workshop
(3) Benchmark
App Run Times
Data rates
read
write
shuffle
Cassandra
Ops/s