Please enable JavaScript.
Coggle requires JavaScript to display documents.
Modelling and Tuning - Coggle Diagram
Modelling and Tuning
Model tuning and performance
Ensure model performance
Select model
LR, Decision Trees, RandomForest,
Ensemble methods: AdaBoost Classifier, Gradient Boosting Classifier, XGBoost Classifier.
Stacking
Run the Model
model =
Model
(random_state=1) -> instantiate
model.fit(X_train,y_train) -> prepare model in training
predict function
**
on validation data -> run model
Evaluate performance
Run metrics according to model
model_score=get_metrics_score(model)
Performance OK?
Test on production data
Overfit, Underfit or Right fit
If Right Fit, the model it likely to
generalize
Tuning
Regularization
Tweak Hyperparameter
1 more item...
Model has to perform well in training, validation and test (production).
validation set
is critical if going to touch hyperparameters
Select right attributes
EDA
Skew analysis
Outliers
any mix up of Gaussians
Feature Engineering
Ensure model is of the right complexity
sweet spot, model is right fit, looping Performance -> Testing -> Tuning -> Performance...
not too simplistic, not too complex
Handling dataset properly
Up sampling / Down sampling
Tweak Hyperparameters
try out Hyp.params according to model
ensure having training, validation and test-production set
Not tune hyperparameters on test/production data... it causes
dataleak
GridSearchCV
RandomSearchCV
Cross Validation
Technique to evaluate a ML model end estimate its performance on unseen data in
production
K Fold
Divides data in k folds using random functions
min k =2, max k = n (LOOCV)
iterate every fold keeping a fold as testing and remaining as training
retain in every loop evaluation score
Average by dividing the sum Scores by k
Analyze average score and dispersion to assess the likely performance of the model in production
LOOCV
Special K Fold case when k=n
How do we measure performance of a model?
Depends on what kind of modeling we are doing
Regression
Rsquare
Adjusted R square
...
Classification
Accuracy
Precision
Recall
F1
...