Question regarding RF weights

[Nathaniel Smoker]

I’m using the random forest classifier in Weka. When the algorithm selects the best feature for a tree node based on the class impurity measure, it calculates the information (entropy) of the class for each value of a categorical feature, in my case I have only binary features. It then computes the class impurity measure based on a weighted sum of those information values, where the weight for each feature value is given by the relative frequency of that feature value in the dataset.

I want to modify the calculation of that weight for each feature value, but I cannot find the part of the code where that weight is calculated.

Can you please tell me which part of the code handles this?

 

 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann]

> I’m using the random forest classifier in Weka. When the algorithm selects the best feature for a tree node based on the class impurity measure, it calculates the information (entropy) of the class for each value of a categorical feature, in my case I have only binary features. It then computes the class impurity measure based on a weighted sum of those information values, where the weight for each feature value is given by the relative frequency of that feature value in the dataset.
>
> I want to modify the calculation of that weight for each feature value, but I cannot find the part of the code where that weight is calculated.
>
> Can you please tell me which part of the code handles this?

RandomForest uses RandomTree as the base classifier for its forest.
Have a look at that class.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
[phone]
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Nathaniel Smoker]

> RandomForest uses RandomTree as the base classifier for its forest.
> Have a look at that class.

 

Thank you for the help Peter, I went back over the code and found it, I must have missed it the first time. If I may ask another question, once I have changed the code, how do I then implement that into the full version of Weka on my system?

 

Thanks,

Nate

From: [hidden email]
Sent: 29 November 2020 02:45
To: [hidden email]
Subject: [Wekalist] Re: Question regarding RF weights

 

> I’m using the random forest classifier in Weka. When the algorithm selects the best feature for a tree node based on the class impurity measure, it calculates the information (entropy) of the class for each value of a categorical feature, in my case I have only binary features. It then computes the class impurity measure based on a weighted sum of those information values, where the weight for each feature value is given by the relative frequency of that feature value in the dataset.
>
> I want to modify the calculation of that weight for each feature value, but I cannot find the part of the code where that weight is calculated.
>
> Can you please tell me which part of the code handles this?

RandomForest uses RandomTree as the base classifier for its forest.
Have a look at that class.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
[phone]
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

[Peter Reutemann]

> > RandomForest uses RandomTree as the base classifier for its forest.
> > Have a look at that class.
>
> Thank you for the help Peter, I went back over the code and found it, I must have missed it the first time.

Cool!

> If I may ask another question, once I have changed the code, how do I then implement that into the full version of Weka on my system?

Every download of Weka comes with a source jar (weka-src.jar) that
contains the complete code and build scripts. Change the RandomTree
code in that source tree and then compile the code base with ant:
https://waikato.github.io/weka-wiki/compiling_weka/

Then just use the weka.jar from the "dist" directory.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
[phone]
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html