Please enable JavaScript.

Coggle requires JavaScript to display documents.

Decision Trees - Coggle Diagram

- - - - There are 2 considerations for an ensemble to work
        
        they should be diverse and complementary to each other
        
        they should be at least better than the random model
      - :question:How they are better than a single DT
        
        do exercise with a model with accuracy of 70% and ensemble of 3 of this models
        
        ensemble with boost up the prediction score, try with probability calculaions
      - Algorithm
        
        sample the data from dataset
        
        create tree by taking the random subset of features
        
        repeat the above steps n time for n Trees
        
        while testing final prediction scores is taken as majority score
      - Advantages
        
        more stable as the results are average out, results in low variance
        
        diversity arises because of randomly taking the subset of features
        
        immune to curse of dimensionality, less computation time
        
        we can paraellilize the tree creation as they are independent
        
        OOB error gives good estimate of model performace on unseen dataset, no need to use extra set for testing
      - Out of Bag Error
        
        essentially like CV score without even explicity doing it
        
        OOB is the mean prediction score on each training sample using only trees which that do not have that data in training
      - Time Taken to build a forest
        
        depends on T(number of trees needed)
        
        depends on S(size of sampled dataset)
        
        F(number of random features taken)
      - HyperParameters
        
        n_estimators - number of DTs to use in forest
        
        Else most of the hyperparameters are related to DTs
        
        Optimisation
        
        optimisation of hyperparameters can be done using GridSearchCv technique