C5.0 Decision Trees in Weka

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

C5.0 Decision Trees in Weka

simone
I know that this question is periodically posted, but it has passed a lot of
time since the last time: maybe there are news.

I'm working on an exam project at university and I have to generate a C5.0
Decision Tree on my data.
Why not C4.5? Because some attributes have a weight specified.

I just need one tree with a limited depth, so things like forests or
algorithms that generate many trees or other things are not useful in my
case.

At the moment I'm using the Decision Tree (Weight-Based) operator in
RapidMiner, but the result is still not what is expected. I found that R
supports C5.0, but it should be better if nothing have to be programmed.

So: is there an implementation of C5.0 in Weka?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Riccardo Granero
J48 is the classifier based on c4.5. You can find a lot of info in the forum. 

Riccardo

Il giorno sab 2 nov 2019 alle 05:00 simone <[hidden email]> ha scritto:
I know that this question is periodically posted, but it has passed a lot of
time since the last time: maybe there are news.

I'm working on an exam project at university and I have to generate a C5.0
Decision Tree on my data.
Why not C4.5? Because some attributes have a weight specified.

I just need one tree with a limited depth, so things like forests or
algorithms that generate many trees or other things are not useful in my
case.

At the moment I'm using the Decision Tree (Weight-Based) operator in
RapidMiner, but the result is still not what is expected. I found that R
supports C5.0, but it should be better if nothing have to be programmed.

So: is there an implementation of C5.0 in Weka?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Riccardo Granero

you can try another decision tree:

RepTree

 Fast decision tree learner. Builds a decision/regression tree using information gain/variance and prunes it using reduced-error pruning (with backfitting). Only sorts values for numeric attributes once. Missing values are dealt with by splitting the corresponding instances into pieces (i.e. as in C4.5).

For this one too you can find all the info in the forum.

Riccardo

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

simone
Thank you, but I need a tree operator to specify some user defined weights
for some specific attributes.



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Michael Hall


On Nov 2, 2019, at 10:38 AM, simone <[hidden email]> wrote:

Thank you, but I need a tree operator to specify some user defined weights
for some specific attributes.


C5.0 I believe is commercial

C4.5 algorithm

Quinlan went on to create C5.0 and See5 (C5.0 for Unix/Linux, See5 for Windows) which he markets commercially. 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Michael Hall


On Nov 2, 2019, at 9:20 PM, Michael Hall <[hidden email]> wrote:



On Nov 2, 2019, at 10:38 AM, simone <[hidden email]> wrote:

Thank you, but I need a tree operator to specify some user defined weights
for some specific attributes.


C5.0 I believe is commercial

C4.5 algorithm

Quinlan went on to create C5.0 and See5 (C5.0 for Unix/Linux, See5 for Windows) which he markets commercially. 

I see that it appears you are correct that someone (Max Kuhn) wrote a C5.0 extension as an R package.
You could probably install the Weka RPlugin package and try it using MLR as I recall?
I need to get that installed on my current computer.



_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Michael Hall


On Nov 2, 2019, at 9:43 PM, Michael Hall <[hidden email]> wrote:



On Nov 2, 2019, at 9:20 PM, Michael Hall <[hidden email]> wrote:



On Nov 2, 2019, at 10:38 AM, simone <[hidden email]> wrote:

Thank you, but I need a tree operator to specify some user defined weights
for some specific attributes.


C5.0 I believe is commercial

C4.5 algorithm

Quinlan went on to create C5.0 and See5 (C5.0 for Unix/Linux, See5 for Windows) which he markets commercially. 

I see that it appears you are correct that someone (Max Kuhn) wrote a C5.0 extension as an R package.
You could probably install the Weka RPlugin package and try it using MLR as I recall?
I need to get that installed on my current computer.


OK, with the Iris dataset.

Learner classif.C50 from package C50
Type: classif
Name: C50; Short name: C50
Class: classif.C50
Properties: twoclass,multiclass,numerics,factors,prob,missings,weights
Predict-Type: response
Hyperparameters: 


Call:
C5.0.default(x = d$data, y = d$target, weights = .weights, control = ctrl)

Classification Tree
Number of samples: 150 
Number of predictors: 4 

Tree size: 4 

Non-standard options: attempt to group attributes



Time taken to build model: 0.42 seconds

=== Stratified cross-validation ===
=== Summary ===

Correctly Classified Instances         144               96      %
Incorrectly Classified Instances         6                4      %
Kappa statistic                          0.94  
Mean absolute error                      0.0483
Root mean squared error                  0.1622
Relative absolute error                 10.8672 %
Root relative squared error             34.4018 %
Total Number of Instances              150     

There were some delays for things to be installed. Not sure how the parameters would work for the weights.

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Eibe Frank-2
Administrator
In reply to this post by simone
Attribute weights are currently not supported by the decision tree learners in WEKA (or the MLRClassifier interface to R schemes that is available through the RPlugin package for WEKA).

However, if you really mean *instance weights*, then, yes, they are supported by most decision tree learners in WEKA, e.g., REPTree, J48, and DecisionStump.

ARFF files allow the specification of instance weights in curly braces. The dataset editor in the Preprocess panel of the Explorer also allows specification of instance weights (and attribute weights, but these are not currently used by the decision tree learners in WEKA).

Cheers,
Eibe

> On 2/11/2019, at 2:25 PM, simone <[hidden email]> wrote:
>
> I know that this question is periodically posted, but it has passed a lot of
> time since the last time: maybe there are news.
>
> I'm working on an exam project at university and I have to generate a C5.0
> Decision Tree on my data.
> Why not C4.5? Because some attributes have a weight specified.
>
> I just need one tree with a limited depth, so things like forests or
> algorithms that generate many trees or other things are not useful in my
> case.
>
> At the moment I'm using the Decision Tree (Weight-Based) operator in
> RapidMiner, but the result is still not what is expected. I found that R
> supports C5.0, but it should be better if nothing have to be programmed.
>
> So: is there an implementation of C5.0 in Weka?
>
>
>
> --
> Sent from: https://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list -- [hidden email]
> Send posts to: To unsubscribe send an email to [hidden email]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Michael Hall


On Nov 6, 2019, at 5:09 PM, Eibe Frank <[hidden email]> wrote:

Attribute weights are currently not supported by the decision tree learners in WEKA (or the MLRClassifier interface to R schemes that is available through the RPlugin package for WEKA).

I was thinking the weights hyperparameter here might be what she meant.

C5.0.default(x = d$data, y = d$target, weights = .weights, control = ctrl)

But I could be wrong and I’m not remembering how the interface passes these hyperparameters.


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: C5.0 Decision Trees in Weka

Eibe Frank-2
Administrator

That parameter in the C5.0 wrapper for R determines “case weights” (i.e., instance weights), not attribute weights, see

 

  https://www.rdocumentation.org/packages/C50/versions/0.1.2/topics/C5.0.default

 

As I said, J48 (as well as REPTree, etc.) will take “case weights” into account if they are stored with the instances in the data (e.g., by being specified as instance weights in the ARFF file).

 

Support for instance weights (in most learning algorithms) is actually one of the strengths of WEKA.

 

Whether a classifier can make use of instance weights can be seen by looking at the capabilities of the classifier (or checking the corresponding Javadoc for whether the classifier implements WeightedInstancesHandler).

 

Cheers,

Eibe

 

From: [hidden email]
Sent: Thursday, 7 November 2019 1:22 PM
To: [hidden email]
Subject: [Wekalist] Re: C5.0 Decision Trees in Weka

 

 



On Nov 6, 2019, at 5:09 PM, Eibe Frank <[hidden email]> wrote:

 

Attribute weights are currently not supported by the decision tree learners in WEKA (or the MLRClassifier interface to R schemes that is available through the RPlugin package for WEKA).

 

I was thinking the weights hyperparameter here might be what she meant.

 

C5.0.default(x = d$data, y = d$target, weights = .weights, control = ctrl)

 

But I could be wrong and I’m not remembering how the interface passes these hyperparameters.

 

 


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html