Clarification on how Weka's ROC curve for AdaBoost is plotted

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Clarification on how Weka's ROC curve for AdaBoost is plotted

obinna Igbe
I have seen other questions in the mailing list archives about this similar question. One being this:
http://comments.gmane.org/gmane.comp.ai.weka/37424

According to Eibe's reply, "When WEKA draws an ROC curve for Adaboost, it treats it just like any other classifier, so only the threshold on the final ensemble prediction is varied." I will like to understand what Eibe means by "threshold on the final ensemble prediction". Is the threshold not say 0 where a test value <0 is labeled -1 and ones >0 are labeled +1. How is this varied?

With the little I know about Ada boost, the final decision is made by taking the sign of summation of all the contribution of the selected weak classifiers (i.e. classifier weight multiplied by its label of the instance which could be +1 or -1). This will lead to say getting the sign of say a vector of this nature [a1*h1(t) + a1*h1(t)+...+ an*hn(t)] where n is the number of selected weak classifiers. So does varying the threshold mean say ranking the above from smallest, then selecting the lowest value in the above vector, and walking my way up to the largest + value while calculating FPR and TPR for these?

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Clarification on how Weka's ROC curve for AdaBoost is plotted

Eibe Frank-3
Check the distributionForInstance(Instance) method from AdaBoostM1 in WEKA to see how exactly it obtains "probability estimates":

https://svn.cms.waikato.ac.nz/svn/weka/trunk/weka/src/main/java/weka/classifiers/meta/AdaBoostM1.java

Given these probability estimates for a particular class for all instances in a test set, the AUROC for that class is calculated in the usual way by ranking the instances according to the probability estimates and proceeding from top to bottom, recording TPR and FPR rates in the process. More info how on this is in our book.

Cheers,
Eibe

On Wed, Jan 4, 2017 at 6:47 PM, obinna Igbe <[hidden email]> wrote:
I have seen other questions in the mailing list archives about this similar question. One being this:
http://comments.gmane.org/gmane.comp.ai.weka/37424

According to Eibe's reply, "When WEKA draws an ROC curve for Adaboost, it treats it just like any other classifier, so only the threshold on the final ensemble prediction is varied." I will like to understand what Eibe means by "threshold on the final ensemble prediction". Is the threshold not say 0 where a test value <0 is labeled -1 and ones >0 are labeled +1. How is this varied?

With the little I know about Ada boost, the final decision is made by taking the sign of summation of all the contribution of the selected weak classifiers (i.e. classifier weight multiplied by its label of the instance which could be +1 or -1). This will lead to say getting the sign of say a vector of this nature [a1*h1(t) + a1*h1(t)+...+ an*hn(t)] where n is the number of selected weak classifiers. So does varying the threshold mean say ranking the above from smallest, then selecting the lowest value in the above vector, and walking my way up to the largest + value while calculating FPR and TPR for these?

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html



_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Loading...