Please enable JavaScript.

Coggle requires JavaScript to display documents.

Data Wrangling, References: Relational Databases for Data Analysis /…

- - - - Why data might be missing:
        
        Systematic causes
        
        Privacy concerns
        
        Information loss
      - Methods to check for missing data:
        
        Verify that the data was uploaded correctly
        
        Review small chunks of the data
        
        Look at the statistics of the entire dataset
      - Types of missing data
        
        Structurally Missing Data: We expect this data to be missing for some logical reason
        
        Missing Completely at Random (MCAR): The probability of any datapoint being MCAR is the same for all data points — this type of missing data is mostly hypothetical
        
        💡 Solution
        
        Missing at Random (MAR): The probability of any data point being MAR is the same within groups of the observed data — this is much more realistic than MCAR
        
        💡 Solution
        
        Missing Not at Random (MNAR): There is some reason why the data is missing
        
        💡 Solution
      - Deletion:
        
        Listwise deletion
        
        Pairwise deletion
        
        Drop variables
      - Single imputation:
        
        Last Observation Carried Forward (LOCF)
        
        Next Observation Carried Backward (NOCB)
        
        Baseline Observation Carried Forward (BOCF)
      - Multiple imputation:
        Use the IterativeImputer module within sklearn
      - 4) Final review / Tidy data
- - - - 📌 Goal
      - Steps