Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Engineering with Azure Synapse Apache Spark Pools - Coggle Diagram
Data Engineering with Azure Synapse Apache Spark Pools
Apache Spark in Azure Synapse Analytics
Azure HDInsight
Azure Databricks
Integrate SQL and Apache Spark pools in Azure Synapse Analytics
Integration methods between SQL and spark pools in Azure Synapse Analytics
Use-cases for SQL and spark pools integration
Authenticate in Azure Synapse Analytics
Transfer data between SQL and spark pool in Azure Synapse Analytics
Authenticate between spark and SQL pool in Azure Synapse Analytics
Transfer data outside the synapse workspace using the PySpark connector
Monitor and manage data engineering workloads with Apache Spark in Azure Synapse Analytics
Monitor spark pools in Azure Synapse Analytics
Base-line Apache Spark performance
Optimize Apache Spark jobs in Azure Synapse Analytics
Choosing the data abstraction
Data Frames
RDDs
Bucketing
Optimize joins and shuffles
Optimize Job Execution
Automate scaling of Apache Spark pools
Ingest data with Apache Spark notebooks in Azure Synapse Analytics
Spark Notebooks
Use-cases for spark notebooks
Exploratory data analysis using a familiar paradigm
Supported languages in spark notebooks
Develop Spark Notebooks
Run Spark Notebooks
Load Data in Spark Notebooks
Save Spark Notebooks
DataFrames in Apache Spark Pools in Azure Synapse Analytics
Load data into a spark dataframe
Flatten nested structures and explode arrays