Weights in Hoeffding Tree

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Weights in Hoeffding Tree

A Nikam
I am working on Hoeffding tree as a base classifier for a boosting algorithm.
While studying the source code in MOA, I could not figure out exactly how the instance weights are being manipulated in the class hierarchy. What is the significance of instance weights in Hoeffding Tree? How and why do they get modified exactly?
Please Help.... Thanks in advance !
Reply | Threaded
Open this post in threaded view
|

Re: Weights in Hoeffding Tree

Albert Bifet
> I am working on Hoeffding tree as a base classifier for a boosting algorithm.
> While studying the source code in MOA, I could not figure out exactly how
> the instance weights are being manipulated in the class hierarchy. What is
> the significance of instance weights in Hoeffding Tree? How and why do they
> get modified exactly?

You may look at the code of OzaBoost

http://www.cs.waikato.ac.nz/~abifet/MOA/API/_oza_boost_8java_source.html

In your boosting method, suppose you have an ensemble, you only have
to create a new instance with the new weight, and train the classifier
desired with this new instance

Instance weightedInst = (Instance) inst.copy();
weightedInst.setWeight(newWeight);
this.ensemble[i].trainOnInstance(weightedInst);

On the Hoeffding Tree

http://www.cs.waikato.ac.nz/~abifet/MOA/API/_hoeffding_tree_8java_source.html

the tree is trained using trainOnInstanceImpl, where the learning
nodes are trained using learnFromInstance:

learningNode.learnFromInstance(inst, this);

In learnFromInstance, the AttributeClassObserver objects are used to
monitor the statistics on the node.

Cheers,

Albert

-----------------------------------------------------------------------------------
Albert Bifet, Dept. of Computer Science, University of Waikato
http://www.cs.waikato.ac.nz/~abifet         +64 7 838 4466 Ext 8766

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.scms.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weights in Hoeffding Tree

whewage
I'm using a hoeffding tree for classification.
weight of each instance is equal to 1.
But when I get the tree model description, weights of the classes in leaf
nodes are double values (eg: 23.012).
Can some one please explain what's happening there.



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weights in Hoeffding Tree

Eibe Frank
This appears to happen for numeric predictor attributes only, i.e., when the Gaussian approximation is used to model the distribution of data in a class. Perhaps Richard Kirkby's PhD thesis has some info on this.

Cheers,
Eibe

On Thu, Feb 27, 2020 at 12:20 PM whewage <[hidden email]> wrote:
I'm using a hoeffding tree for classification.
weight of each instance is equal to 1.
But when I get the tree model description, weights of the classes in leaf
nodes are double values (eg: 23.012).
Can some one please explain what's happening there.



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Weights in Hoeffding Tree

whewage
Hi,
I went through the thesis you suggested.
It seems like, when the gaussian approximation done for classes in leaf
nodes, weight of the examples are defined by area under the gaussian curve.
But still it's not clear to me.



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html