Random Classifier

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Random Classifier

Riccardo Granero
Hi, I need compare the classifier result with
 a random classifier. In particular I would compare
the selected classifier with a random classifier in the experimenter.
Until now I did it out of weka.


Many thanks for the attention.

Riccardo Granero




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Random Classifier

Eibe Frank-3
The kappa statistic will tell you how your classifier compares to a random classifier that assigns the same number of instances to each class as the classifier that is being evaluated (i.e., the confusion matrix of the random classifier has the same marginal column and row totals as the confusion matrix of the classifier being evaluated). .

A kappa value of zero means your classifier does not improve on the corresponding random classifier. A kappa value of one means that your classifier is perfect. Negative values of kappa are also possible.

Kappa is one of the metrics output by WEKA, including the Experimenter.

Cheers,
Eibe

On Sun, Dec 29, 2019 at 7:23 AM Riccardo Granero <[hidden email]> wrote:
Hi, I need compare the classifier result with
 a random classifier. In particular I would compare
the selected classifier with a random classifier in the experimenter.
Until now I did it out of weka.


Many thanks for the attention.

Riccardo Granero




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Random Classifier

Riccardo Granero
Thanks Eibe,
Then to verify if the result of the selected classifier is random or not
I could select the kappa in the experimenter
and then verify the significance level with
the t-test of the zero rule (baseline scheme) classifier and the selected
classifier.
Is it right?

Thanks
Riccardo



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Random Classifier

Eibe Frank-3
Yes, that sounds fine. ZeroR will always give you a kappa of 0.

Cheers,
Eibe

On Mon, Dec 30, 2019 at 7:56 AM Riccardo Granero <[hidden email]> wrote:
Thanks Eibe,
Then to verify if the result of the selected classifier is random or not
I could select the kappa in the experimenter
and then verify the significance level with
the t-test of the zero rule (baseline scheme) classifier and the selected
classifier.
Is it right?

Thanks
Riccardo



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Random Classifier

Riccardo Granero

Thanks Eibe!

Happy new year to you and all the precious staff who support us.

Riccardo


Happy new year Eibe 

Il giorno mer 1 gen 2020 alle 02:52 Eibe Frank <[hidden email]> ha scritto:
Yes, that sounds fine. ZeroR will always give you a kappa of 0.

Cheers,
Eibe

On Mon, Dec 30, 2019 at 7:56 AM Riccardo Granero <[hidden email]> wrote:
Thanks Eibe,
Then to verify if the result of the selected classifier is random or not
I could select the kappa in the experimenter
and then verify the significance level with
the t-test of the zero rule (baseline scheme) classifier and the selected
classifier.
Is it right?

Thanks
Riccardo



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html