Filter didn't make the test instance immediately available!

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Filter didn't make the test instance immediately available!

David
I have a problem with appling a resample filter and test it to a dataset
different than the testing set. I have no problem with appling a resample
filter and test it with ten-fold. It seems like the training dataset when
sampled changed the number of features, strange. See below the code and the
error message.

/*
 *  How to use WEKA API in Java
 *  Copyright (C) 2014
 *  @author Dr Noureddin M. Sadawi ([hidden email])
 *  
 *  This program is free software: you can redistribute it and/or modify
 *  it as you wish ...
 *  I ask you only, as a professional courtesy, to cite my name, web page
 *  and my YouTube Channel!
 *  
 */

//import required classes
import weka.core.Instances;


import java.util.Random;
import weka.classifiers.Evaluation;
import weka.classifiers.meta.FilteredClassifier;
import weka.classifiers.trees.J48;
import weka.filters.supervised.instance.Resample;
import weka.core.converters.ConverterUtils.DataSource;
import weka.classifiers.evaluation.*;




public class TestWekaSampling{
        public static void main(String args[]) throws Exception{
                //load datasets
                                DataSource source1 = new DataSource("C:/Program
Files/Weka-3-8/data/breast-cancerKnown.arff");
                                Instances training = source1.getDataSet();
                                DataSource source2 = new DataSource("C:/Program
Files/Weka-3-8/data/breast-cancerNOTK.arff");
                                Instances testing = source2.getDataSet();
                               

                               
                                int numAttr = training.numAttributes();
                                training.setClassIndex(numAttr - 1);
                                testing.setClassIndex(numAttr - 1);

                               

                                J48 j48 = new J48();

                                j48.buildClassifier(training);


                                Evaluation eval = new Evaluation(testing);
                                eval.evaluateModel(j48, testing); //not sampled
                                //eval.crossValidateModel(j48, testing, 10, new Random(1)); it works
                               
                                System.out.println("AUC = "+eval.areaUnderROC(1));
                                System.out.println("kappa = "+eval.kappa());
                               
                               
                    Resample resample = new Resample();
                                resample.setInputFormat(training);
                                FilteredClassifier fc = new FilteredClassifier();

                               
                               
                                fc.setClassifier(j48);
                                fc.setFilter(resample);
                               
                               
                                Evaluation eval2 = new Evaluation(testing);
                                // eval2.crossValidateModel(fc, testing, 10, new Random(1)); it works
                                eval2.evaluateModel(fc, testing); //sampled
                               
                                System.out.println("AUC = "+eval2.areaUnderROC(1));
                                System.out.println("kappa = "+eval2.kappa());
                               
                               
        }
}


/******************** OUTPUT ****************/
AUC = 0.5895522388059702
kappa = 0.16449885233358863
Exception in thread "main" java.lang.Exception: Filter didn't make the test
instance immediately available!
        at
weka.classifiers.meta.FilteredClassifier.filterInstance(FilteredClassifier.java:796)
        at
weka.classifiers.meta.FilteredClassifier.distributionForInstance(FilteredClassifier.java:825)
        at
weka.classifiers.evaluation.Evaluation.evaluationForSingleInstance(Evaluation.java:2192)
        at
weka.classifiers.evaluation.Evaluation.evaluateModelOnceAndRecordPrediction(Evaluation.java:2230)
        at
weka.classifiers.evaluation.Evaluation.evaluateModel(Evaluation.java:2106)
        at weka.classifiers.Evaluation.evaluateModel(Evaluation.java:689)
        at TestWekaSampling.main(TestWekaSampling.java:69)




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Filter didn't make the test instance immediately available!

Peter Reutemann
> I have a problem with appling a resample filter and test it to a dataset
> different than the testing set. I have no problem with appling a resample
> filter and test it with ten-fold. It seems like the training dataset when
> sampled changed the number of features, strange. See below the code and the
> error message.

Your code was a bit strange, as you combined a not-really initialized
Resample filter and a built J48 within a non-built FilteredClassifier.
I've simplified and corrected your code. See attachment.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

TestWekaSampling.java (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Filter didn't make the test instance immediately available!

David
This is excellent, thanks!!!
Do you know where it is possible to find small weka examples?



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Filter didn't make the test instance immediately available!

Peter Reutemann-3
On February 3, 2020 10:22:36 PM GMT+13:00, David <[hidden email]> wrote:

>This is excellent, thanks!!!
>Do you know where it is possible to find small weka examples?
>
>
>
>--
>Sent from: https://weka.8497.n7.nabble.com/
>_______________________________________________
>Wekalist mailing list -- [hidden email]
>Send posts to: To unsubscribe send an email to
>[hidden email]
>To subscribe, unsubscribe, etc., visit
>https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
>List etiquette:
>http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

Have you checked the example archive in your Weka installation? Also, the manual and the wiki explain how to use the API.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html