Computing external classification accuracy metrics?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Computing external classification accuracy metrics?

David
Suppose I have a prediction from a model outside WEKA. Is there a way I can
compute the WEKA accuracy metrics of this model? I.e., I should be able to
feed WEKA with the predicted and actual values and apply some sort of no
prediction. The reason would be to easily compare the WEKA predictions to
this external prediction. Yes I could compute the metrics of the external
model by myself but the devil is in the details...



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Computing external classification accuracy metrics?

Eibe Frank-2
Administrator

If you have a single test set for which you want performance statistics, you can achieve what you want by using appropriately structured training and test data files together with IBk. An example is given below. The train.arff file has the class values that we want to predict. The test.arff file has the actual class values.

 

The idea is to use IBk to implement a form of lookup table: the first attribute in the data indicates the value to look up.

 

Cheers,

Eibe

 

$ more train.arff test.arff

::::::::::::::

train.arff

::::::::::::::

@relation train

 

@attribute ID numeric

@attribute class {c1, c2, c3}

 

@data

1, c1

2, c3

3, c2

5, c2

6, c1

::::::::::::::

test.arff

::::::::::::::

@relation train

 

@attribute ID numeric

@attribute class {c1, c2, c3}

 

@data

1, c2

2, c3

3, c2

5, c1

6, c3

$ java weka.Run .IBk -t train.arff -T test.arff -classifications .PlainText

 

 

=== Predictions on test data ===

 

    inst#     actual  predicted error prediction

        1       2:c2       1:c1   +   0.75

        2       3:c3       3:c3       0.75

        3       2:c2       2:c2       0.75

        4       1:c1       2:c2   +   0.75

        5       3:c3       1:c1   +   0.75

 

From: [hidden email]
Sent: Thursday, 20 December 2018 8:52 AM
To: [hidden email]
Subject: [Wekalist] Computing external classification accuracy metrics?

 

Suppose I have a prediction from a model outside WEKA. Is there a way I can

compute the WEKA accuracy metrics of this model? I.e., I should be able to

feed WEKA with the predicted and actual values and apply some sort of no

prediction. The reason would be to easily compare the WEKA predictions to

this external prediction. Yes I could compute the metrics of the external

model by myself but the devil is in the details...

 

 

 

--

Sent from: http://weka.8497.n7.nabble.com/

_______________________________________________

Wekalist mailing list

Send posts to: [hidden email]

To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist

List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html