Prediction output consisting only of question marks

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Prediction output consisting only of question marks

rmsmaby
I'm using a training and a test dataset with the random forest algorithm for this project. My issue is that the output prediction shows the total number of instances is 0, and the detailed accuracy by class information is all represented by question marks.

 Some additional information: I get no pop-up errors for this, and all my attributes in the data files match between the training and test file (as does the spelling of the attribute names). I've also tried running the test file with cross-validation, and that has worked fine. The issue is just occurring when I run the training dataset using the 'supplied test set' option to make predictions with my test dataset. I'm fairly new to using Weka, so any advice would be greatly appreciate!
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Prediction output consisting only of question marks

Peter Reutemann
> I'm using a training and a test dataset with the random forest algorithm for this project. My issue is that the output prediction shows the total number of instances is 0, and the detailed accuracy by class information is all represented by question marks.
>
>  Some additional information: I get no pop-up errors for this, and all my attributes in the data files match between the training and test file (as does the spelling of the attribute names). I've also tried running the test file with cross-validation, and that has worked fine. The issue is just occurring when I run the training dataset using the 'supplied test set' option to make predictions with my test dataset. I'm fairly new to using Weka, so any advice would be greatly appreciate!

Can you verify this behaviour with another dataset? E.g., split the
iris UCI dataset and perform the same procedure.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Prediction output consisting only of question marks

rmsmaby
Thanks for your reply! And yes, I found some step by step instructions with a training and test data from a university course online (https://users.cs.northwestern.edu/~ddowney/courses/349_Spring2017/pset1.html) and ran through the process. I got the same results as I did with my research data (detailed accuracy by class output is all question marks), making me think that the issue is with something that I'm doing rather than with my research data training and test setup.
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Prediction output consisting only of question marks

Peter Reutemann
> Thanks for your reply! And yes, I found some step by step instructions with a training and test data from a university course online (https://users.cs.northwestern.edu/~ddowney/courses/349_Spring2017/pset1.html) and ran through the process. I got the same results as I did with my research data (detailed accuracy by class output is all question marks), making me think that the issue is with something that I'm doing rather than with my research data training and test setup.

Not sure how these datasets were prepared.

Here is how you can split a dataset into a train/test set by using
randomized subsets:
1. Load the dataset (eg iris.arff) in the Weka Explorer
2. Use the following Resample filter setup (change the percentage if
you want to)
  weka.filters.unsupervised.instance.Resample -S 1 -Z 66.0 -no-replacement
3. Apply the filter and save the dataset as "train.arff"
4. Undo the filter changes (to get the original dataset back again)
5. Open the filter properties and set "invertSelection" to "True"
(then we get the remainder of the dataset)
6. Apply the filter and save the dataset as "test.arff"

For using this dataset split:
1. Load the "train.arff" file
2. Go to the Classify tab
3. Select RandomForest
4. Check "Supplied test set" and select the "test.arff" through the dialog
5. Click on Start

With the iris dataset, I get something like this:

Correctly Classified Instances          48               94.1176 %
Incorrectly Classified Instances         3                5.8824 %
Kappa statistic                          0.9115
Mean absolute error                      0.0424
Root mean squared error                  0.1873
Relative absolute error                  9.4585 %
Root relative squared error             39.3672 %
Total Number of Instances               51

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html