test of a unknown dataset with saved model

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

test of a unknown dataset with saved model

cedkhader

Hi all

 

I created a dataset which shows excellent performance when used with Random Forest or Random Trees Algorithm. I checked this model using 10-fold Validation and also using the 60% / 40%  “percentage split method”.

 

That’s all OK and shows the detection matrix and other details of the classification ,.. etc.

 

But the problem is when I load a new test set for testing the just created model. In this case, I use the first dataset for training and another completely unknown dataset for testing. In such cases, I don’t get any results as the detection matrix, though I loaded the saved model .Under the classification results,  I get “? ? ? ? . . . ” for TP, FP , .. and for weights.

 

Please advise, because I think, it is not clear how to create the test set, and which steps are needed for testing completely unknow datasets.

 

P.S : My datasets have the same structure of relation name and attributes names and types.

 

I replace the last attribute in test set with “?”, but it does not work, too.

 

 

Regards

 

 

Derar

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: test of a unknown dataset with saved model

Peter Reutemann
> I created a dataset which shows excellent performance when used with Random Forest or Random Trees Algorithm. I checked this model using 10-fold Validation and also using the 60% / 40%  “percentage split method”.
>
>
>
> That’s all OK and shows the detection matrix and other details of the classification ,.. etc.
>
>
>
> But the problem is when I load a new test set for testing the just created model. In this case, I use the first dataset for training and another completely unknown dataset for testing. In such cases, I don’t get any results as the detection matrix, though I loaded the saved model .Under the classification results,  I get “? ? ? ? . . . ” for TP, FP , .. and for weights.
>
>
>
> Please advise, because I think, it is not clear how to create the test set, and which steps are needed for testing completely unknow datasets.
>
>
>
> P.S : My datasets have the same structure of relation name and attributes names and types.
>
>
>
> I replace the last attribute in test set with “?”, but it does not work, too.

In order to obtain statistics on your test data, you need to have
actual class values present (your "ground truth"). Otherwise, you can
only make predictions, but not evaluations (that's why TP, FP etc are
all "?").

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html