Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Engineer - Coggle Diagram
Data Engineer
Work Activities
Data Wrangling
Data Gathering
Assess Data
Data Cleaning
Data Enrichment
Data Transformation
Testing
unit tests
integration tests
performance tests
quality tests
Data Pipeline Building
alerts
logs
orchestration
choreography
fall backs
roll backs
reruns
notifications
incremental
snaps
streams
batches
migrations
historical
Exploratory Data Analisis
Data Profiling
Outliers
Dependencies
Mappings
Rules
meassures of spread
measures of center
shape of data
normal distribution
Data Quality Checks
Tuning
Query Performance
Resources
Tools and Technologies
Version Control System
git
Deployment
Environments
UAT
Dev
QA
Stagging
Prod
Platform
hadoop stack
spark
Third party tools
tableau
power bi
Kowledge Domains
Data Management DMBOK2
Data Architecture
Architectures
hub and spoke
bus
independent data marts
centralized
federated
layers/tiers
bottom tier DWH Server
middle tier OLAP Server
top tier FE Tools
Data Modeling and Design
Conceptual, Logical & Physical
Normal Forms
UNF Unnormal Form
1NF First Normal Form
2NF Second Normal Form
3NF Third Normal Form
Physical Models
Flat File
Dimensional Models
Star Schema
Snowflake Schema
Data Storage
SQL
NoSQL
csv, json, xml
Data Integration & Interoperability
ETL
ELT
Data Warehousing & Business Intelligence
dimensions
facts
data lake
data swamp
data mart
data warehouse
granularity
Data Quality
Clarity
Accesibility
Accuracy
Relevance
Completeness
Timeliness
Profiling
8 Vs
Programming Language
Paradigms
OOP
Clases & Objects
Methods
Types
Access Modifiers
Interface (Abstraction)
Loops
Functional
Anonymous Functions (Lambdas)
Higher Order Functions and Curries
functions as return values
recursion
pure functions
thread safe parallelism
Immutability
Scala
sbt / maven
tail recursion
scalatest
Collections
Sequences
Tuples and Maps
generics
partially applied functions
Spark (lib)
PLSQL
Visualization
KPI
slicers
drill up / drill down
Dashboard types
Operational
Analytical
Strategic
Chart types
univariate
bars
donut
histograms
bivariate
scatter
heatmap
boxplot
cluster bars
line plots
radar / spider