doubt about

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

doubt about

andria lan

hi  all, 

1- Is there any relation between the “prediction margin” (found in “Visualize classifier errors”) and actual scores for each instance (“Predictions on test data” that is produced using “Output prediction”)?

2- Is it possible to convert “prediction margin” into actual scores for each instance?


Thank you.


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: doubt about

Eibe Frank-2
Administrator
Let’s assume c is the index of the class label associated with an instance. Let p(j) be the classifier’s estimated class probability for the class label with index j. Then WEKA’s GUI’s and evaluation methods define the margin as p(c) - max_{j != c} p(j).

In the best case, the margin is 1 because the estimated probability for the actually observed class label p(c) is 1 and all other probabilities are 0. In the worst case, the margin is -1. This happens when probability 1 is associated with an incorrect class label.

If you output the full estimated class probability distribution for each test instance when specifying output of predictions under “More options…” in the Classify Panel, you will be able to compute the margin for each test instance yourself.

Another place where the margin is used in the GUI is when you output the cumulative margin distribution  by selecting “Visualize margin curve” from the pop-up menu that can be triggered by right-clicking in the Result list of the Classify Panel.

Note that there are other definitions of the term margin in the literature. WEKA’s way of defining the margin in terms of estimated probabilities is just one option.

Cheers,
Eibe

> On 20/09/2019, at 7:13 PM, Andria Lan <[hidden email]> wrote:
>
> hi  all,
>
> 1- Is there any relation between the “prediction margin” (found in “Visualize classifier errors”) and actual scores for each instance (“Predictions on test data” that is produced using “Output prediction”)?
>
> 2- Is it possible to convert “prediction margin” into actual scores for each instance?
>
>
>
> Thank you.
>
> _______________________________________________
> Wekalist mailing list -- [hidden email]
> Send posts to: To unsubscribe send an email to [hidden email]
> To subscribe, unsubscribe, etc., visit
> https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html