Please enable JavaScript.
Coggle requires JavaScript to display documents.
Chapter 21-23 (Chapter 21 set up prediction system (Choose deployment…
Chapter 21-23
Chapter 21 set up prediction system
need to implement the new system
Retraining model
recall the possibility that the model developed in DataRobot may not perform quite as well as expected when put into practice as suggested by the holdout sample LogLoss score being slightly lower that of the cross validation sample
success rate can be lower is because even deteriorating over time, us that the environment being modelled changes subtly as time passes
you can use 100% of the data for the model since validation and overfitting isnt a problem
Choose deployment strategy
drag and drop scoring
is accessed through the predict screen of the selected model
API scoring
is relatively straight forward for those able to program in R or Python. have to write a program that uploads new patient data to the API
Data Robot Prime Scoring
creates an approximation of the selected model, available as code in the Python and Java programming languages prime scoring is availability based on DR account type
Batch Scoring
uses DR API to upload and score multiple large files in parallel
In place scoring
allows for exporting the selected model as an executable file to be used in an apache spark environment
Chapter 23 Create Model Monitoring and Maintenance Plan
potential problems
related to documentation of the newly installed system, it is also necessary to create a monitoring and maintenance plan
serves to inform others what to do in the event of changes in the environment that stand to impact the effectiveness of the model
environmental changes may appear will making a new model
Strategies
DR will fail rather than attempt to make the available data
avoid failure
it is good idea to return the model as soon as sufficient new data is available
wait till target value is available
Chapter 22 Document Modeling Process for Reproducibility
Model documentation
where most projects often fail
when you can do more of the desirable central machine learning work
Will also help to save time in the future
specify all steps you have taken in your ML process