Algorithms that support giving weight to attributes?

classic Classic list List threaded Threaded
12 messages Options
JC
Reply | Threaded
Open this post in threaded view
|

Algorithms that support giving weight to attributes?

JC
Hi everyone!

I'm working right now with *Naive Bayes because I know that this algorithm
support giving weight to attributes but I would like to know if there are
other algorithms that support it too*.

Now, *in my code I change Naive Bayes for Random Tree (I want to use this
algorithm) and I get the following error*:

/weka.core.WekaException: weka.classifiers.trees.RandomTree: Some attribute
weights are not equal to 1 and scheme does not implement the
WeightedAttributesHandler interface!
        at weka.core.Capabilities.test(Capabilities.java:1228)
        at weka.core.Capabilities.test(Capabilities.java:1138)
        at weka.core.Capabilities.testWithFail(Capabilities.java:1468)
        at weka.classifiers.trees.RandomTree.buildClassifier(RandomTree.java:731)
        at
com.example.RandomTreeWeights.trainRandomCommittee(RandomTreeWeights.java:522)
        at com.example.RandomTreeWeights.naiveExe(RandomTreeWeights.java:290)
        at com.example.RandomTreeWeights.main(RandomTreeWeights.java:962)

/
Exactly, can I give weights to attributes using Random Tree? Should I have
to implement first /WeightedAttributesHandler/ ? and finally can I use
another algorithm like Naive Bayes directly without use
WeightedAttributesHandler?

Thanks in advance!



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank
Currently, only NaiveBayes and FilteredClassifier implement WeightedAttributesHandler.

To implement WeightedAttributesHandler in FilteredClassifier if the base learner is not a WeightedAttributesHandler (i.e., currently, if it is neither NaiveBayes nor another FilteredClassifier), attributes are sampled with replacement by normalising the weights into a probability distribution for sampling before the data is passed to the base learner.

So, if you want to apply some learning algorithm X to weighted data, you could wrap it into FilteredClassifier. Obviously, a single random subsets of attributes won't reflect the attribute weights very accurately, but then you could in turn wrap FilteredClassifier into RandomCommittee, which will give you an ensemble classifier where a different set of attributes is sampled according to the weights for each ensemble member. You will loose any interpretability of the model but it seems to me that this should work reasonably well if you use a sufficient number of ensemble members in RandomCommittee. (Alternatively, you could use Bagging instead of RandomCommittee. In that case, you would get both resampling of instances and resampling of attributes.)

Cheers,
Eibe

On Wed, Sep 25, 2019 at 9:20 PM JC <[hidden email]> wrote:
Hi everyone!

I'm working right now with *Naive Bayes because I know that this algorithm
support giving weight to attributes but I would like to know if there are
other algorithms that support it too*.

Now, *in my code I change Naive Bayes for Random Tree (I want to use this
algorithm) and I get the following error*:

/weka.core.WekaException: weka.classifiers.trees.RandomTree: Some attribute
weights are not equal to 1 and scheme does not implement the
WeightedAttributesHandler interface!
        at weka.core.Capabilities.test(Capabilities.java:1228)
        at weka.core.Capabilities.test(Capabilities.java:1138)
        at weka.core.Capabilities.testWithFail(Capabilities.java:1468)
        at weka.classifiers.trees.RandomTree.buildClassifier(RandomTree.java:731)
        at
com.example.RandomTreeWeights.trainRandomCommittee(RandomTreeWeights.java:522)
        at com.example.RandomTreeWeights.naiveExe(RandomTreeWeights.java:290)
        at com.example.RandomTreeWeights.main(RandomTreeWeights.java:962)

/
Exactly, can I give weights to attributes using Random Tree? Should I have
to implement first /WeightedAttributesHandler/ ? and finally can I use
another algorithm like Naive Bayes directly without use
WeightedAttributesHandler?

Thanks in advance!



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

JC
Thank you Eibe for your response.

Perfect, FilteredClassifier works fine!
By default FilteredClassifier uses J48, right? Where can I find information
about FilteredClassifier?

*And if I would prefer to use another algorithm * ... So if I have
understood it well, I would have to do this:

- Create a FilteredClassifier
- Create RandomTree inside the FilteredClassifier by this way:

/ FilteredClassifier cls = new FilteredClassifier();
                cls.setClassifier(new RandomTree());/

- But how can I create a RandomComittee and wrap the FilteredClassifier with
it?

*If I do this, is different use a RandomTree than a RandomTree wrapped by
FilteredClassifier?*
Are there any examples of this? because this sounds hard to implement it
being my first time using RandomComittee and FilteredClassifier.



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank-2
Administrator
Implement it in the same way as you are setting the base classifier in FilteredClassifier.

Note that you should also set the filter in FilteredClassifier to AllFilter so that no filtering (other than attribute subsampling) is performed.

Cheers,
Eibe

> On 27/09/2019, at 12:08 AM, JC <[hidden email]> wrote:
>
> Thank you Eibe for your response.
>
> Perfect, FilteredClassifier works fine!
> By default FilteredClassifier uses J48, right? Where can I find information
> about FilteredClassifier?
>
> *And if I would prefer to use another algorithm * ... So if I have
> understood it well, I would have to do this:
>
> - Create a FilteredClassifier
> - Create RandomTree inside the FilteredClassifier by this way:
>
> / FilteredClassifier cls = new FilteredClassifier();
> cls.setClassifier(new RandomTree());/
>
> - But how can I create a RandomComittee and wrap the FilteredClassifier with
> it?
>
> *If I do this, is different use a RandomTree than a RandomTree wrapped by
> FilteredClassifier?*
> Are there any examples of this? because this sounds hard to implement it
> being my first time using RandomComittee and FilteredClassifier.
>
>
>
> --
> Sent from: https://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list -- [hidden email]
> Send posts to: To unsubscribe send an email to [hidden email]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

JC
If I set FilteredClassifier to AllFilter I can't use the method
cls.setClassifier(new RandomTree()); to use the classifier that I want to
use because AllFilter have not this method.

How could I do this so?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank-3
I meant that you should use the setFilter() method of FilteredClassifier to specify AllFilter as the filter that FilteredClassifier should apply.

Cheers,
Eibe

On Fri, 27 Sep 2019 at 9:19 PM, JC <[hidden email]> wrote:
If I set FilteredClassifier to AllFilter I can't use the method
cls.setClassifier(new RandomTree()); to use the classifier that I want to
use because AllFilter have not this method.

How could I do this so?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

JC
This post was updated on .
Thank you again Eibe.

I doing this:

/ RandomCommittee rct = new RandomCommittee();
                AllFilter af = new AllFilter();
                FilteredClassifier cls = new FilteredClassifier();
                cls.setFilter(af);
                cls.setClassifier(new RandomTree());
                rct.setClassifier(cls);/

and then I run the code but all I got is this:

/** Random Tree Evaluation with Datasets **

Correctly Classified Instances         801      100%
Incorrectly Classified Instances          0       0%
Kappa statistic                                    1    
Mean absolute error                           0    
Root mean squared error                   0    
Relative absolute error                       0%
Root relative squared error                0%
Total Number of Instances                 801    
/

How can be possible this? Am I doing something wrong??



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- wekalist@list.waikato.ac.nz
Send posts to: To unsubscribe send an email to wekalist-leave@list.waikato.ac.nz
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank
This result makes sense assuming you are evaluating on the training data. RandomTree builds an unpruned tree.

Cheers,
Eibe

On Sat, Sep 28, 2019 at 9:20 PM JC <[hidden email]> wrote:
Thank you again Eibe.

I doing this:

/               RandomCommittee rct = new RandomCommittee();
                AllFilter af = new AllFilter();
                FilteredClassifier cls = new FilteredClassifier();
                cls.setFilter(af);
                cls.setClassifier(new RandomTree());
                rct.setClassifier(cls);/

and then I run the code but all I got is this:

/** Random Tree Evaluation with Datasets **

Correctly Classified Instances         801      100%
Incorrectly Classified Instances          0       0%
Kappa statistic                                    1     
Mean absolute error                           0     
Root mean squared error                   0     
Relative absolute error                       0%
Root relative squared error                0%
Total Number of Instances                 801     
/

How can be possible this? Am I doing something wrong?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

JC
This post was updated on .
Thanks again Eibe for your time.

I have done this but I have the same result as if I use the AllFilters
filter than if I do not use it.

Should it be a different result or the same?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- wekalist@list.waikato.ac.nz
Send posts to: To unsubscribe send an email to wekalist-leave@list.waikato.ac.nz
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank-2
Administrator
By default, FilteredClassifier will apply the Discretize filter. If all your attributes are nominal, this will have the same effect as using the AllFilter.

Cheers,
Eibe

> On 1/10/2019, at 10:09 PM, JC <[hidden email]> wrote:
>
> Thanks again Eibe for your time.
>
> I have done this but I have the same result as if I use the AllFilters
> filter than if I do not use it.
>
> Should it be a different result or the same?
>
>
>
> --
> Sent from: https://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list -- [hidden email]
> Send posts to: To unsubscribe send an email to [hidden email]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
JC
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

JC
All my attributes are numerical except one that is nominal.

Maybe have I something wrong in my code? I have it like this:

/ RandomCommittee rct = new RandomCommittee();
                AllFilter af = new AllFilter();
                FilteredClassifier cls = new FilteredClassifier();
                cls.setFilter(af);
                cls.setClassifier(new RandomTree());
                rct.setClassifier(cls);/

Thank you again!



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Algorithms that support giving weight to attributes?

Eibe Frank-3
That looks fine to me.

Cheers,
Eibe

On Wed, 2 Oct 2019 at 10:13 PM, JC <[hidden email]> wrote:
All my attributes are numerical except one that is nominal.

Maybe have I something wrong in my code? I have it like this:

/               RandomCommittee rct = new RandomCommittee();
                AllFilter af = new AllFilter();
                FilteredClassifier cls = new FilteredClassifier();
                cls.setFilter(af);
                cls.setClassifier(new RandomTree());
                rct.setClassifier(cls);/

Thank you again!



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html