Please enable JavaScript.
Coggle requires JavaScript to display documents.
TDS ML Ops Notes - Coggle Diagram
TDS ML Ops Notes
Model Unit Testing
Pytest style code tests
Auto verification check during retraining for goodness of fit improvement
Third party tools to automate model testing
Model Governance
permission settings in ML Flow
model testing / validation check
Verify model tested against a synthetic dataset with a known signal
A/B Testing
Add test of statistical significance for to catch prediction changes
Add practical tests to determine if business improvements have been made based on data
validate new model works better than old model
Development Platform
Infra as Code
User Policies
Monitoring
Prometheus
Grafana
Drift Detection
investigate Seldon-core
alibi-detect
Interpretability
alibi-detect
what artifacts should be create to increase model understanding and interpretability
Outlier Detection
alibi-detect
Adversarial Attack Detection
alibi-detect
https://towardsdatascience.com/get-started-with-mlops-fd7062cab018