Please enable JavaScript.

Coggle requires JavaScript to display documents.

Applied Machine Learning (Methods (Decision Tree (Pre-pruning (max_depth…

- - - - Training Data
  - - - Supervised
        
        Regression
        
        Values
        
        Classification
        
        Categories
      - Unsupervised
        
        Clustering
        
        Outlier detection
    - - Representation
        
        Evaluation
        
        Optimization
  - - - ML
    - - Data Manipulation
        
        import pandas as pd
    - - Representation
        
        import matplotlib.pyplot as plt
    - - import numpy as np
    - - Scientific Computing Tools
        
        import scipy as sp
  - - - fruits.head()
    - - y = fruits['fruit_label']
    - - 75/25
    - - missing
      - inconsistent
    - - knn.fit(X_train, y_train)
- - - - Weighting
      - Aggregation method
      - k
        
        5 by default
      - Distance
        
        Euclidian by default
    - - Easy
  - - - Minimize
        
        Mean Squared Error
      - No hyperparameter
    - - W = Feature Weights
      - b = Intercept
      - Loss function
        
        target
    - - Least Square
        
        PLUS alpha * sum(Wj²)
        
        L2 Regularization
        
        Alpha = hyperparameter
      - Lof of variables
        
        Small effect
    - - Least Square
        
        PLUS alpha * sum(abs(Wj))
        
        L1 Regularization
        
        Sparse Solution
        
        0 for least influence
      - A few variables
        
        Large effect
    - - New features
        
        Polynomial combination
        
        Initial features
      - Easier classification
      - Overfitting :warning:
        
        High degree
  - - - PLUS non-linear function [0-1]
        
        Activation
    - - Target
    - - C hyperparameter
        
        C Low
        
        Small W
    - - Recommanded
    - - [-1,1] prediction
        
        Sign
      - Classifier margin
        
        Max Width
        
        Data point hit
        
        Max Classifier
        
        Best width
        
        SVM = Support Vector Machine
        
        Large C
        
        Less regularization
        
        Less error tolerant
    - - Same principles
        
        For each class
  - - - scaler = MinMaxScaler()
      - X_train_scaled = scaler.transform(X_train)
  - - - Split
    - - On Top
    - - Regression
    - - max_depth
        
        min_samples_leaf
        
        max_leaf_nodes
      - Overfitting avoid
    - - No normalization
    - - Generalization issues
        
        Ensemble of trees
  - - - To apply Linear SVM
        
        In Transformed Feature Space
    - - Default in SVC
      - Gamma = Hyperparameter
        
        High Gamma
        
        Close
        
        To be similar
        
        Low Gamma
        
        Smoother boundaries
    - - Not good on large data sets
        
        Difficult to interpret
  - - - cross_val_score
      - Stratified
        
        Mix
        
        Class proportion
      - validation_curve
- - - - Inputs
  - - - Too specific
    - - Too simple
- - - - Imbalanced classes
      - Partial picture
    - - Amount of revenue
    - - null metric
      - DummyClassifier
        
        Meand
        
        Median
        
        Quantile
    - - True Positive & Negative
        
        False Positive & Negative
      - Confusion Matrix
        
        confusion_matrix(y_test , y_predicted)
  - - - (TP+TN)/Total
    - - (FP+FN)/Total
    - - TP / (TP + FN)
      - Large TPR
        
        Large False Positive
        
        Tumor detection
      - Recall
        
        recall_score
    - - TP / (TP + FP)
      - Document Classification
        
        Customer facing
      - precision_score
    - - FP / (TN + FP)
    - - 2 X Precision X TPR / (Precision + TPR)
        
        2 TP / (2 TP + FN + FP)
      - f1_score
    - - (1+B²) Precision TPR / (B² * Precision + TPR)
        
        Precision with B=0.5
        
        Recall with B = 2
  - - - Threshold
        
        For positive or negative
    - - class probability
  - - - FPR X TPR
  - - - Binary per class
    - - Predicted Digit X True Digit
    - - Equal class weight
    - - Equal instance weight
    - - average = 'micro'
  - - - mean_absolute_error
      - mean_squared_error
      - median_absolute_error
  - - - Validation
        
        Test
- - - - Gaussian
        
        GaussianNB()
        
        Continuous
      - Bernouilly
        
        Binary
      - Multinomial
        
        Discrete features
        
        Word cound
    - - No correlation
    - - SImple & efficient
        
        Good baseline model
  - - - Better generalization
      - More stable
      - Less overfitting
      - Diverse Trees
    - - Regression
        
        RandomForestRegressor
        
        Prediction mean
      - Classification
        
        RandomForestClassifier
        
        Average probability
        
        Larger probability / class
    - - Number of trees
    - - Input randomness
    - - features subset
    - - No preprocessing needed
      - Prediction
      - Parallelization
    - - :red_cross:High Dimensional Features
      - Interpretation :question:
    - - Reproductible
  - - - Weak learners
        
        Mistake corrections
    - - Number of trees
    - - Level of correction
        
        On previous trees
    - - 3 to 5 typically
    - - Memory
        
        Fast
      - Accuracy
    - - Interpretation :question:
      - Parameter tuning
      - :no_entry: High dimensional features
      - Computation :warning:
  - - - Logistic regressions
    - - Relu
        
        Default
      - Tanh
      - Logistic
    - - alpha
        
        High alpha
        
        Less overfitting
    - - Significant training
        
        Similar feature types
  - - - Features
        
        Anonymization
      - Training data
    - - High correlation :warning:
      - High weigths :warning:
    - - Timestamp cutoff
      - Final test set
      - Data protection
        
        Each cross-validation fold
        
        separately
- - - - Structure
      - Information
  - - - kmeans = KMeans(n_clusters = 3)
        
        kmeans.fit(X)
      - Simple cluster
        
        Same size
    - - cls = AgglomerativeClustering(n_clusters = 3)
        
        cls.fit_predict(X)
      - Linkage criteria
    - - No # of clusters
      - DBSCAN(eps = 2, min_samples = 2)
      - dbscan.fit_predict(X)
  - - - Continuous probability
    - - Better visualization
        
        Compression
      - PCA
        
        Principal Component Analysis
        
        PCA(n_components = 2).fit(X_normalized)
        
        pca.transform(X_normalized)
        
        Projection
        
        Best component
      - Manifold learning
        
        MultiDimensional Scaling (MDS)
        
        Distance preserving
        
        Low-dimesion projection
        
        MDS(n_components = 2)
        
        mds.fit_transform(X_fruits_normalized)
        
        StandardScaler().fit(X_fruits).transform(X_fruits)
        
        t-SNE
        
        2D Projection
        
        Preserving neighbors
        
        TSNE(random_state = 0)
        
        tsne.fit_transform(X_fruits_normalized)