ClassBalancer + CostSensitive classification

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

ClassBalancer + CostSensitive classification

Marina Santini
Hi,

I would like to ask a few questions about possible solutions to solve the unbalanced dataset problem.

1) Is it possible to use the MetaCost classifier from any of the Weka GUIs? (MetaCost: https://weka.wikispaces.com/MetaCost).
I did not find it in Explorer.

2) Is it a good idea to apply the ClassBalancer filter and a CostSensitive Classifier on the same dataset? I cannot see any side effects, but I wish to be sure I did not miss anything.

3) I would like to apply a filter and a costSensitive  classifier with the wrapper method. 
This means that in the Classify panel I select:
meta->FilteredClassifier;
then I specify costSensitive-->RandomForest (in the classifier box)
then I select ClassBalancer (in the Filter box).

I get the following error: 

Inline images 1

What does it mean? How can I fix it?

When I apply the ClassBalancer filter from the Preprocess tab and then a cost-Sensitive classifier from the Classify tab, everything goes smoothly. 


Thanks in advance for your answer, 

Cheers, Marina


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: ClassBalancer + CostSensitive classification

Eibe Frank-2
Administrator

> On 26/05/2017, at 4:08 AM, Marina Santini <[hidden email]> wrote:
>
> 1) Is it possible to use the MetaCost classifier from any of the Weka GUIs? (MetaCost: https://weka.wikispaces.com/MetaCost).
> I did not find it in Explorer.

It’s in a package:

http://weka.sourceforge.net/packageMetaData/metaCost/index.html

> 2) Is it a good idea to apply the ClassBalancer filter and a CostSensitive Classifier on the same dataset? I cannot see any side effects, but I wish to be sure I did not miss anything.

I would not use them together. It’s best to simply adjust the cost matrix for CostSensitiveClassifier.

By default, CostSensitiveClassifier does is internal reweighting based on the cost matrix anyway. Also, if you switch to the minimum expected cost approach in CostSensitiveClassifier, it assumes accurate class probability estimates, but if you modify the weights with ClassBalancer, the class probability estimates will be biased.

> 3) I would like to apply a filter and a costSensitive  classifier with the wrapper method.
> This means that in the Classify panel I select:
> meta->FilteredClassifier;
> then I specify costSensitive-->RandomForest (in the classifier box)
> then I select ClassBalancer (in the Filter box).
>
> I get the following error:
>
> <image.png>
>
> What does it mean? How can I fix it?
>
> When I apply the ClassBalancer filter from the Preprocess tab and then a cost-Sensitive classifier from the Classify tab, everything goes smoothly.

Hmm, this works fine for me in WEKA 3.8.1/3.9.1. Anyway, you don’t really need the combination (see above).

Cheers,
Eibe

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html