Please enable JavaScript.
Coggle requires JavaScript to display documents.
Q&A DELTA - Coggle Diagram
Q&A DELTA
Spark transformation
best way to think of Framework is as a "CONNECTOR"
ETL SCRIPTS
PYTHON
SQL
ORCHESTRATION
PARALLELIZATION
MULTIPROCESSING
RDD PARALLELIZATION
THREADING
S/W ENG LOAD,COMPLEXITY
IDF takeaways
FR
advantage of running Spark under Databricks vs without
CODING LOAD
HOW TO MINIMIZE DEPENDENCY
Advantage of Framework over IDF, ADF, AWS Glue, etc.
THREADING/MULTIPROC LOAD
NFR
SW ENGINEERING NEED
IDF SIMILARITY