Getting empty results with test dataset

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Getting empty results with test dataset

asadbtk
I am using train dataset and train it "Using Training set", then save the model, then load the saved model and provide the test set and re-run the classifier. The problem is that I get completely empty result i.e. total number of instances=0, Accuracy=?, Precision=?, F-meaure=? etc. 

My train and test datasets have the same features except the values of Class variable in test set are replaced from Y/N to the ? symbol.. The attribute name Play {'Y',N} is same both in train and test datasets but only the values in test dataset are replaced from Y/N to ?. 

This is the first time I am using Weka using a separate train and test data. Earlier I just used the k-fold cross-validation and things were quite simple.

Regards  

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Getting empty results with test dataset

Peter Reutemann-3
On July 16, 2019 4:17:01 AM GMT+12:00, javed khan <[hidden email]> wrote:

>I am using train dataset and train it "Using Training set", then save
>the
>model, then load the saved model and provide the test set and re-run
>the
>classifier. The problem is that I get completely empty result i.e.
>total
>number of instances=0, Accuracy=?, Precision=?, F-meaure=? etc.
>
>My train and test datasets have the same features except the values of
>Class variable in test set are replaced from Y/N to the ? symbol.. The
>attribute name Play {'Y',N} is same both in train and test datasets but
>only the values in test dataset are replaced from Y/N to ?.
>
>This is the first time I am using Weka using a separate train and test
>data. Earlier I just used the k-fold cross-validation and things were
>quite
>simple.
>
>Regards

If your test set has no class values (ground truth), how is Weka supposed to calculate statistics? It has nothing to compare the predictions against...

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Getting empty results with test dataset

asadbtk
Hi Peter, but isn't it true that we need to replace the values of class variable with some vague values i.e. ? or 1, or A etc. This is what I have heard so far about test set.

On Mon, Jul 15, 2019 at 9:30 PM Peter Reutemann <[hidden email]> wrote:
On July 16, 2019 4:17:01 AM GMT+12:00, javed khan <[hidden email]> wrote:
>I am using train dataset and train it "Using Training set", then save
>the
>model, then load the saved model and provide the test set and re-run
>the
>classifier. The problem is that I get completely empty result i.e.
>total
>number of instances=0, Accuracy=?, Precision=?, F-meaure=? etc.
>
>My train and test datasets have the same features except the values of
>Class variable in test set are replaced from Y/N to the ? symbol.. The
>attribute name Play {'Y',N} is same both in train and test datasets but
>only the values in test dataset are replaced from Y/N to ?.
>
>This is the first time I am using Weka using a separate train and test
>data. Earlier I just used the k-fold cross-validation and things were
>quite
>simple.
>
>Regards

If your test set has no class values (ground truth), how is Weka supposed to calculate statistics? It has nothing to compare the predictions against...

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Getting empty results with test dataset

Peter Reutemann-3
On July 16, 2019 8:21:47 PM GMT+12:00, javed khan <[hidden email]> wrote:
>Hi Peter, but isn't it true that we need to replace the values of class
>variable with some vague values i.e. ? or 1, or A etc. This is what I
>have
>heard so far about test set.
>
>
No.

Weka automatically removes the class value before obtaining a prediction for an instance.

For statistics you require ground truth. How else would Weka determine these?

The only time you unset the class is when using the API and presenting an instance to a model (what Weka does automatically when evaluating against a test set).

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Getting empty results with test dataset

asadbtk
Ok I got it Peter.. Thanks for the useful information. 

On Tue, Jul 16, 2019 at 12:03 PM Peter Reutemann <[hidden email]> wrote:
On July 16, 2019 8:21:47 PM GMT+12:00, javed khan <[hidden email]> wrote:
>Hi Peter, but isn't it true that we need to replace the values of class
>variable with some vague values i.e. ? or 1, or A etc. This is what I
>have
>heard so far about test set.
>
>
No.

Weka automatically removes the class value before obtaining a prediction for an instance.

For statistics you require ground truth. How else would Weka determine these?

The only time you unset the class is when using the API and presenting an instance to a model (what Weka does automatically when evaluating against a test set).

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html