Regression with M5p

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Regression with M5p

Gaetano
Hi
I have a test set and a training set and I should perform a numeric prediction of the class label using M5p.
How it is possible to invoke m5p using java weka api?
I tried this but I think that it doesn't work because I get values very high of rmse:
("testSet" have class values equal to '0' while "testset" no)

M5P  predictor = new M5P ();
predictor.buildClassifier(trainingSet);
for (int i = 0; i < testSet.numInstances(); i++)
{
    double pred = predictor.classifyInstance(testSet.instance(i));
    Attribute last = testSet.instance(i).classAttribute();
    testSet.instance(i).setValue(last, pred);
    Evaluation eval = new Evaluation(testSet);
    eval.evaluateModelOnce(pred, testset.instance(i));
    double rmse = eval.rootMeanSquaredError();
}

If it is wrong can you advise me which instruction to use to make numerical prediction?
Thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Eibe Frank-2
Administrator
Why are you setting the class value of the test instance to be the predicted value? That doesn't make sense. The error should always be zero in this case.

Cheers,
Eibe

> On 3/06/2017, at 4:07 AM, Gaetano <[hidden email]> wrote:
>
> Hi
> I have a test set and a training set and I should perform a numeric
> prediction of the class label using M5p.
> How it is possible to invoke m5p using java weka api?
> I tried this but I think that it doesn't work because I get values very high
> of rmse:
> ("testSet" have class values equal to '0' while "testset" no)
>
> M5P  predictor = new M5P ();
> predictor.buildClassifier(trainingSet);
> for (int i = 0; i < testSet.numInstances(); i++)
> {
>    double pred = predictor.classifyInstance(testSet.instance(i));
>    Attribute last = testSet.instance(i).classAttribute();
>    testSet.instance(i).setValue(last, pred);
>    Evaluation eval = new Evaluation(testSet);
>    eval.evaluateModelOnce(pred, testset.instance(i));
>    double rmse = eval.rootMeanSquaredError();
> }
>
> If it is wrong can you advise me which instruction to use to make numerical
> prediction?
> Thanks
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Regression-with-M5p-tp40847.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Gaetano
Because i have create a second test set where I inserted instead of class attribute the predicted value.
But I changed in this way:

for (int i = 0; i < testset.numInstances(); i++)
{
   double pred = predictor.classifyInstance(testset.instance(i));
   Evaluation eval = new Evaluation(trainingSet);
   eval.evaluateModelOnce(pred, testset.instance(i));
   double rmse = eval.rootMeanSquaredError();
}

Unfortunately I get very high rmse values and can not understand what the cause is.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Eibe Frank-2
Administrator
You are calculating the RMSE for each individual test instance, not the test set as a whole, because a new evaluation object is created for every instance. Is that intentional?

Regardless, your use of M5P looks fine to me. Do you have reason to believe that the RMSE should not be so large (i.e., results obtained using some other method)?

One scenario where M5P can fail badly is when you have test instances that are outside the domain of the training data. M5P uses linear regression models at the leaf nodes of the tree and if linear extrapolation is not appropriate, you can get very large errors.

Cheers,
Eibe

> On 3 Jun 2017, at 20:19, Gaetano <[hidden email]> wrote:
>
> Because i have create a second test set where I inserted instead of class
> attribute the predicted value.
> But I changed in this way:
>
> for (int i = 0; i < testset.numInstances(); i++)
> {
>   double pred = predictor.classifyInstance(testset.instance(i));
>   Evaluation eval = new Evaluation(trainingSet);
>   eval.evaluateModelOnce(pred, testset.instance(i));
>   double rmse = eval.rootMeanSquaredError();
> }
>
> Unfortunately I get very high rmse values and can not understand what the
> cause is.
>
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Regression-with-M5p-tp40847p40859.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Gaetano
Hi Eibe
I would like to ask you to make numerical prediction for each instance, the only available statement is "classifyInstance" istruction (that i write before)?
Because afterwards, when I calculate the rmse, I get very low values (example 2.325554E-5) and I can not understand the reason.
Thanks
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Eibe Frank-2
Administrator
The classifyInstance() method is the correct method to use. Why is a small error problematic?

Cheers,
Eibe

> On 18/06/2017, at 8:49 PM, Gaetano <[hidden email]> wrote:
>
> Hi Eibe
> I would like to ask you to make numerical prediction for each instance, the
> only available statement is "classifyInstance" istruction (that i write
> before)?
> Because afterwards, when I calculate the rmse, I get very low values
> (example 2.325554E-5) and I can not understand the reason.
> Thanks
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Regression-with-M5p-tp40847p40980.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Gaetano
Yes in my case I should get values around 0.1
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Regression with M5p

Eibe Frank-2
Administrator
You’ll probably have to inspect the model tree and derive the corresponding prediction manually from the tree to see what’s going on.

Cheers,
Eibe

> On 19/06/2017, at 7:19 PM, Gaetano <[hidden email]> wrote:
>
> Yes in my case I should get values around 0.1
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Regression-with-M5p-tp40847p40987.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Loading...