Micro/Macro-Averging

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Micro/Macro-Averging

mcbenly
Hi,
I understand, micro and macro averaging are good performance evaluater for
multi-class problem. (See my confusion matrix result in attachment).  

Is there any option in Weka to see either of those measure result in the
output?

I saw  'unweighted_micro_avg_F_measure'  in 'experimenter'. But not sure
what that is, and how its being calculated.

OR is it something I will need to manually calculate myself?


Thanks,  <https://weka.8497.n7.nabble.com/file/t6937/weka.jpg>



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Micro/Macro-Averging

Eibe Frank-2
Administrator

For multi-class problems, micro-averaged F-measure is actually the same as classification accuracy:

 

  https://waikato.github.io/weka-blog/posts/2019-02-16-micro_average/

 

WEKA’s output, in the Explorer and at the command-line, will show you, for each class, the F-measure for that class (obtained by treating that class as the positive class and the union of the other classes as the negative class).

 

WEKA will also show you the weighted average of these per-class F-measures. The weight for a class C is set to N(C) / N, where N(C) is the number of test instances in class C  and N is the total number of test instances.

 

The Explorer and the command-line interface do not output an unweighted average of the per-class F-measures.

 

In my view, in most cases,  AUROC or AUPRC are preferable to F-measure when comparing models. The F-measure is based on a single point contained in the recall-precision curve and the classifier is not optimised for this particular point. AUPRC, which WEKA also outputs, considers all points along the curve and is thus more robust.

 

If you really want to use F-measure, make sure you wrap all algorithms you are comparing into WEKA’s ThresholdSelector meta classifier and make sure that it is set to optimise F-measure. However, this is only applicable to datasets with two classes.

 

Cheers,

Eibe

 

From: [hidden email]
Sent: Saturday, 31 August 2019 9:53 AM
To: [hidden email]
Subject: [Wekalist] Micro/Macro-Averging

 

Hi,

I understand, micro and macro averaging are good performance evaluater for

multi-class problem. (See my confusion matrix result in attachment). 

 

Is there any option in Weka to see either of those measure result in the

output?

 

I saw  'unweighted_micro_avg_F_measure'  in 'experimenter'. But not sure

what that is, and how its being calculated.

 

OR is it something I will need to manually calculate myself?

 

 

Thanks,  <https://weka.8497.n7.nabble.com/file/t6937/weka.jpg>

 

 

 

--

Sent from: https://weka.8497.n7.nabble.com/

_______________________________________________

Wekalist mailing list

Send posts to: [hidden email]

To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist

List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Micro/Macro-Averging

mcbenly
Thanks Eibe, really helpful.

Thanks,




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html