Please enable JavaScript.
Coggle requires JavaScript to display documents.
Azure Integration and ETL - Coggle Diagram
Azure Integration and ETL
Azure Data Factory (ADF)
: A fully managed data integration service that allows you to create, schedule, and orchestrate ETL and data movement workflows at scale.
Features
Data Movement: Supports over 90 built-in connectors to on-premises and cloud data sources.
Data Transformation: Use data flows for code-free data transformation, or integrate with Azure Databricks and HDInsight for more complex transformations.
Scheduling and Orchestration: Create pipelines that orchestrate the movement and transformation of data.
Monitoring and Management: Provides monitoring and logging capabilities to track pipeline execution.
Uses
Building data pipelines for analytics and machine learning
Data migration and consolidation
Data warehousing
Azure Databricks
: An Apache Spark-based analytics platform optimized for Azure, providing a collaborative environment for data engineers, data scientists, and analysts.
Featurers
Interactive Workspaces: Collaborative notebooks for real-time code development and data exploration.
Delta Lake: Reliable data lake with ACID transactions and scalable metadata handling.
Integration: Seamless integration with Azure services like ADF, Azure Synapse Analytics, and Azure Machine Learning.
Performance: Optimized runtime for high-performance Spark jobs.
Uses
Large-scale data processing
Advanced analytics and machine learning
Real-time stream processing