Quality control and audit steps of the ML models

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Quality control and audit steps of the ML models

neha.bologna
Hello everyone, 

In my project, I evaluated the performance of k-fold CV (i.e. 3, 5 and 10 fold), percentage split and leave one out cross validation in Weka. I get different results with different data and classifiers (e.g. SVM). I submitted my project and my advisor asked me a strange question. He says if an organization deploys your approach, how to make sure it will pass the quality audit performed by the quality assurance team?

Could you plz guide me how we can assess our ML model steps according to the industry quality control and audit? Any guidance or appropriate link to the literature would be highly appreciated.

Warm regards

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Quality control and audit steps of the ML models

Peter Reutemann
> In my project, I evaluated the performance of k-fold CV (i.e. 3, 5 and 10 fold), percentage split and leave one out cross validation in Weka. I get different results with different data and classifiers (e.g. SVM). I submitted my project and my advisor asked me a strange question. He says if an organization deploys your approach, how to make sure it will pass the quality audit performed by the quality assurance team?
>
> Could you plz guide me how we can assess our ML model steps according to the industry quality control and audit? Any guidance or appropriate link to the literature would be highly appreciated.

Using the Experimenter, you can perform repeated cross-validations
(but differently seeded), which will give you mean and standard
deviation for your statistics.
It is up to the company/organization to decide whether these error
margins are deemed good enough or not.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html