Please enable JavaScript.
Coggle requires JavaScript to display documents.
Data Transfer - Coggle Diagram
AWS Data Sync
- Primarily focuses on transferring data between on-premises and AWS service: S3, EFS, FSx
- Preserves metadata and permissions
- Built-in scheduled transfer support
AWS DataSync Agent
- VM deployed into the on-premise infrastructure
- Can connect with the NFS and SMB of on-prem
- Connects with AWS Data Sync to transfer data between on-premise and AWS infrastructure
AWS Transfer
- Fully managed file transfer (FTP) into/out of S3/EFS
- Has built-in credential support or can be integrated with other authentication systems (AD, LDAP, Okta, Cognito, etc.)
Public Endpoint
- Endpoint exposed to the internet
- Can't filter by source IP
VPC Endpoint (internal)
- Security group or NACL can be used to control access
VPC Endpoint (\w external access)
- Same as internal version of VPC Endpoint, except an Elastic IP is attached
- Security group or NACL can be set to filter access from internet
Snowball Family
- Dedicate hardware support for physical data transfer
Hardwares
- Approx 7 days to transfer
Snowball Edge
- Dedicated hardware that is able to transfer large amount of data physically in or out of AWS
Edge Computing
- While data transfer is the main purpose, it has compute capability
- Use service like AWS IoT Greengrass to run EC2 / Lambda in it
- Compute Optimized: 52vCPUs with 208GiB of RAM
- Storage Optimized: Up to 80TB of HDD storage
AWS Snoecone
- Has built-in DataSync installed, so if online, can connect to AWS
- Also possible to physically send it to AWS offline
- Has Edge Computing capability (2 CPUs, 4GB memory)
AWS Snowmobile
- Extremely large data transfer (exabytes = 1,000PB = 1,000,000TB)
- Each snowmobile can transfer 100PB at once
- Each snowmobile can work in parallel
AWS Data Pipeline
- Automate data movement between different AWS services
- Preserves metadata and permissions
- Support scheduled transfer
- Provides graphical interface