M5P Model Tree: Attribute Selection

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

M5P Model Tree: Attribute Selection

Bill Bane
Hello.

I am building several regression models, with a primary emphasis on understanding the contributions of specific attributes, as opposed to using primarily for prediction.

In one example, the M5P model tree algorithm provides a single branch with one linear model (with very high correlation to actual).  However, the attribute set selected for this model differs greatly from the attribute sets selected by the Linear Regression classifier (using either the M5 or the Greedy attribute selection options of that algorithm, or using various wrapper selection methods for linear regression).

The M5P attribute set is very appealing, based on my domain knowledge, but I don't want to fool myself into confirming my preconceptions!  Can you describe (or point to a description of) the M5P attribute selection approach, and how it conceptually differs from the Greedy or M5 approaches in the ridge regression algorithm?

Sincerely,

--Bill

Reply | Threaded
Open this post in threaded view
|

Re: M5P Model Tree: Attribute Selection

Eibe Frank-2
Administrator
Before building a linear model for a node N (note that building this linear model involves its own “M5P attribute selection” step), M5P reduces the set of predictor attributes available at that node to only include those that are used to make split decisions in the nodes *below* node N, i.e., those nodes that are used to make split decisions in the two subtrees attached to node N.

Note that the final models that are output by M5P are the “smoothed” leaf node models (unless you have turned smoothing off). Smoothing produces a linear combination of all linear models along the path from the corresponding leaf node to the root node of the tree.

Cheers,
Eibe

> On 12/02/2017, at 5:02 AM, Bill Bane <[hidden email]> wrote:
>
> Hello.
>
> I am building several regression models, with a primary emphasis on
> understanding the contributions of specific attributes, as opposed to using
> primarily for prediction.
>
> In one example, the M5P model tree algorithm provides a single branch with
> one linear model (with very high correlation to actual).  However, the
> attribute set selected for this model differs greatly from the attribute
> sets selected by the Linear Regression classifier (using either the M5 or
> the Greedy attribute selection options of that algorithm, or using various
> wrapper selection methods for linear regression).
>
> The M5P attribute set is very appealing, based on my domain knowledge, but I
> don't want to fool myself into confirming my preconceptions!  Can you
> describe (or point to a description of) the M5P attribute selection
> approach, and how it conceptually differs from the Greedy or M5 approaches
> in the ridge regression algorithm?
>
> Sincerely,
>
> --Bill
>
>
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/M5P-Model-Tree-Attribute-Selection-tp39399.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html