Please enable JavaScript.

Coggle requires JavaScript to display documents.

ENSEMBLE TECHNIQUES leverage the "low computational time"…

- - - - Random Forests
        
        Bagging of Decision Trees but taking a subset
        of 'm' columns on each split decision (m < M)
        
        More robust to change of data than DT and overfitting
        
        Hyperparameter Tuning
        
        base_estimator: The base estimator to fit on random subsets of the dataset. If None(default), then the base estimator is a decision tree.
        
        n_estimators: The number of trees in the forest, default = 100.
        
        max_features: The number of features to consider when looking for the best split.
        
        bootstrap: Whether bootstrap samples are used when building trees. If False, the entire dataset is used to build each tree, default=True.
        
        1 more item...
      - Bagging Classifiers
        for a group of one or multiple models: Linear, Logistic regressions, Decision Trees
  - - - Gradient Boosting
        
        By fitting new models to the residuals, the overall learner gradually improves in areas where residuals are initially high•
        
        predict the log of odds instead of the actual labels (0 or 1)
        
        Prediction from previous model + Learning rate * New predictions
        
        GBM for Regression
        
        GBM for Classification
        
        Hyperparameter Tuning
        
        Most of the hyperparameters available are same as random forest classifier.
        
        init: An estimator object that is used to compute the initial predictions. If ‘zero’, the initial raw predictions are set to zero. By default, a DummyEstimator predicting the classes priors is used.
        
        There is no class_weights parameter in gradient boosting.
      - XG Boost
        Extreme Gradient boosting
        
        Parallel Implementation, Cross-Validation, Cache Optimization, Distributed Computation5
        
        Gain calculated from similarity score to build a tree
        
        Hyperparameter Tuning
        
        scale_pos_weight: Control the balance of positive and negative weights, useful for unbalanced classes. It has range from 0 to ∞
        
        subsample: Corresponds to the fraction of observations (the rows) to subsample at each step. By default it is set to 1 meaning that we use all rows.
        
        colsample_bytree Corresponds to the fraction of features (the columns) to use.
        
        colsample_bylevel The subsample ratio of columns for each level.
        
        colsample_bynode The subsample ratio of columns for each node (split).
        
        max_depth: is the maximum number of nodes allowed from the root to the farthest leaf of a tree.
        
        1 more item...
      - AdaBoosting
        Adaptive Boosting
        
        the weightage of data points that were predicted wrong is increased
        
        consists of under fitted models
        
        weighted method of voting
        
        Hyperparameter Tuning
        
        base_estimator: The base estimator from which the boosted ensemble is built.
        By default the base estimator is a decision tree with max_depth=1
        
        n_estimators: The maximum number of estimators at which boosting is terminated. Default value is 50.
        
        learning_rate: Learning rate shrinks the contribution of each classifier by learning_rate. There is a trade-off between learning_rate and n_estimators. Usually n_estimators should be balanced with learning_rate, if one is high, the other should be small.
- - - - Model
        
        abc = model(random_state=1)
        
        abc.fit(X_train,y_train)
        
        Run metrics and Confusion Matrix
        
        abc_score=get_metrics_score(abc)
        
        make_confusion_matrix(abc,y_test)
        
        Hyperparameter Tuning
        
        For for each model
        run GridSearchCV,
        metrics and Confusion Matrix
        
        Run feature importances
        
        Compare all models