Please enable JavaScript.

Coggle requires JavaScript to display documents.

Data Reduction and Splitting (Sampling (machine learning (random sampling…

- - - - apply a unique function (one that keeps only unique rows, discarding any row containing data already encountered), retaining only the first row
        
        You can use the summarize function - each unique "id" can be grouped with each unique configuration of "" into discreet buckets
- - - - data can be filtered into two separate tables (containing the same columns but different rows)
- - - - one value is underrepresented relative to the other in what are called binary targets (two target values, for example “yes” and “no”)
    - - The available data is almost always randomly sampled into several groups in order to evaluate success of a model.
        
        The holdback sample is the first set of data that extracted: used for the final evaluation of the model(s) produced
        
        most often 20% of the available data (for the AutoML tool used in this book)
        
        Think of this data as being locked into a safe for examination at the very end of the project to evaluate the true accuracy of the top-scoring machine- learning model.
        
        the data in the holdout sample remains hidden and locked away. Once the final business decisions are ready to be made using the model, this data can be released and used to evaluate the expected model performance. Before releasing the holdback sample for evaluation, other samples will be used for validating models
      - Step 1: Randomize the order of the data and select a holdout sample
        
        Step 2: Randomize the order of the data and select a holdout sample
        
        Step 3: only occurs if cross-validation is deemed appropriate and valuable for a given model. In AutoML, this will often happen automatically for the top models if the dataset is below a certain size. This step and the following
        (steps 2–5) are conducted tandem as a single process, cross-validation.
        
        Step 4: follows the same actions in step 2. The validation sample is moved down one more step
        
        Step 5: the validation sample is moved down
        
        Step 6: the validation sample is moved down
        
        1 more item...