Please enable JavaScript.

Coggle requires JavaScript to display documents.

초격차 데이터 엔지니어링 - Coggle Diagram

- - - - extract data from different sources(APIs, sensor data, marketing tools, transaction database, ...)
        
        different methods
        
        Partial Extraction
        
        if the source system notifies data change
        
        Partial Extraction
        
        even if no notification, some source systems point changed data and provide extract of such records
        
        Full extract
        
        if no notification nor extract, we need full copy of last extract in the same format to identify the changes
    - - transform data into a usables and trusted resource
        
        data gets cleansed, mapped and transformed, often to a specific schema for its operational needs. using stagind database is common. this step ensures quick roll back.
    - - load the data into the systems. so end users can access and use downstream.
        
        process of writing converted data from a staging area to a target database