Please enable JavaScript.

Coggle requires JavaScript to display documents.

Data Preprocessing - Coggle Diagram

- - - - integrate metadata from different sources
      - entity identification problem: identify real world entities from multiple data sources, e.g., A.cust-id  B.cust-#
    - - Redundant data occur often when integration of multiple databases
        
        Object identification: The same attribute or object may have different names in different databases
        
        Derivable data: One attribute may be a “derived” attribute in another table, e.g., annual revenue
      - Redundant attributes may be able to be detected by correlation analysis and covariance analysis
    - - for the same real world entity, attribute values from different sources are different
      - possible reasons: different representations, different scales, e.g., metric vs. British units
- - - - Age=“42” Birthday=“03/07/1997”
      - Was rating “1,2,3”, now rating “A, B, C”
      - Discrepancy between duplicate records
- - - - z-score normalization
      - normalization by decimal scaling
      - min-max normalization
- - - - but closely maintains the integrity of the original data
      - mining on the reduced dataset should be more efficient yet produce the same analytical results.
      - much smaller in volume