Weighted area under ROC

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Weighted area under ROC

pmichel
Dear Eibe,
I'm working with Weka Experimenter on a multi-class evaluation for different
classifiers. The whole feature set and the class is discretised into 5
states.
One parameter I want to look at is the weighted area under ROC (WAUC?).
Concerning this parameter, I have two questions I’m struggling with:
1. Could help me with a source for the WAUC, where I can cite a compact
equation from? I only found one source from Jialiang Li et al. 2010
(https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4129959/), but I don’t know if
this is right or if there is a better source.
2. Is it right to make the statement, that the WAUC can be considered as
skewness indicator for the target parameter (class)?

I would highly appreciate your help.
Best,
Philipp




--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weighted area under ROC

Eibe Frank-2
Administrator
1. That’s not the right reference for weighted AUC in WEKA. Take a look at

"ROC graphs: Notes and practical considerations for researchers” by Tom Fawcett (2004)

The weighted multi-class AUC is given as the first equation in Section 7.3.2. The paper refers to it as Provost and Domingo’s formulation.

2. No, I don’t think the weighted AUC in WEKA can be considered as a skewness indicator.

Cheers,
Eibe

> On 21/02/2018, at 2:06 AM, pmichel <[hidden email]> wrote:
>
> Dear Eibe,
> I'm working with Weka Experimenter on a multi-class evaluation for different
> classifiers. The whole feature set and the class is discretised into 5
> states.
> One parameter I want to look at is the weighted area under ROC (WAUC?).
> Concerning this parameter, I have two questions I’m struggling with:
> 1. Could help me with a source for the WAUC, where I can cite a compact
> equation from? I only found one source from Jialiang Li et al. 2010
> (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4129959/), but I don’t know if
> this is right or if there is a better source.
> 2. Is it right to make the statement, that the WAUC can be considered as
> skewness indicator for the target parameter (class)?
>
> I would highly appreciate your help.
> Best,
> Philipp
>
>
>
>
> --
> Sent from: http://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weighted area under ROC

pmichel
Dear Eibe,
thanks you so much! This is a great source and also very good to
read/understand.
Best regards,

Philipp



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html