True_positive/True_negative definition in the output report of Experimenter

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

True_positive/True_negative definition in the output report of Experimenter

Shu-Ju Tu
Hi Weka users and experts,

I am having a question of 2-class classification regarding the
definition of true_positive/true_negative definition in Weka.

In Explorer, the prediction of class a/b is listed in the confusion matrix.
You can clearly see the true_positive for class a and true_positive for class b.

In Experimenter, the output lists several quantities, including
true_positive (TP), false_positive (FP), true_negative (TN), and
false_negative (FN).

I am not sure the true_positive value is for class a or class b in the
Experimenter output.
I think the true_positive value for class a is referred as the
true_negative for class b.

Can someone kindly let us know the definition when you can get a chance.

Thanks,
Shu-Ju
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: True_positive/True_negative definition in the output report of Experimenter

Eibe Frank-2
Administrator
By default, the Experimenter calculates those statistics by treating the first class as the positive class.

You can change this by switching to the "Advanced" mode and changing the value of the "classForIRStatistics" parameter of the "ClassifierSplitEvaluator" used by the "CrossValidationResultProducer" (or whichever other result producer you happen to use).

The "classForIRStatistics" parameter has default value 0, which refers to the first class (indexing is 0-based for this particular parameter).

Cheers,
Eibe

> On 24 May 2017, at 15:59, Shu-Ju Tu <[hidden email]> wrote:
>
> Hi Weka users and experts,
>
> I am having a question of 2-class classification regarding the
> definition of true_positive/true_negative definition in Weka.
>
> In Explorer, the prediction of class a/b is listed in the confusion matrix.
> You can clearly see the true_positive for class a and true_positive for class b.
>
> In Experimenter, the output lists several quantities, including
> true_positive (TP), false_positive (FP), true_negative (TN), and
> false_negative (FN).
>
> I am not sure the true_positive value is for class a or class b in the
> Experimenter output.
> I think the true_positive value for class a is referred as the
> true_negative for class b.
>
> Can someone kindly let us know the definition when you can get a chance.
>
> Thanks,
> Shu-Ju
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: True_positive/True_negative definition in the output report of Experimenter

Shu-Ju Tu
Hi Eibe,

Thank you for taking time to answer the question.

What is the exact meaning for the first class as the positive class?

Does that mean the class assigned to the very first instance?

Or the class "a" as appeared in the output of Explorer?

If I want to double check it with Explorer/Experimenter, will the
result of Experimenter be the same as in the output of Explorer if we
set the repetition to 1?

Thanks,
Shu-Ju



2017-05-24 16:18 GMT+08:00 Eibe Frank <[hidden email]>:

> By default, the Experimenter calculates those statistics by treating the first class as the positive class.
>
> You can change this by switching to the "Advanced" mode and changing the value of the "classForIRStatistics" parameter of the "ClassifierSplitEvaluator" used by the "CrossValidationResultProducer" (or whichever other result producer you happen to use).
>
> The "classForIRStatistics" parameter has default value 0, which refers to the first class (indexing is 0-based for this particular parameter).
>
> Cheers,
> Eibe
>
>> On 24 May 2017, at 15:59, Shu-Ju Tu <[hidden email]> wrote:
>>
>> Hi Weka users and experts,
>>
>> I am having a question of 2-class classification regarding the
>> definition of true_positive/true_negative definition in Weka.
>>
>> In Explorer, the prediction of class a/b is listed in the confusion matrix.
>> You can clearly see the true_positive for class a and true_positive for class b.
>>
>> In Experimenter, the output lists several quantities, including
>> true_positive (TP), false_positive (FP), true_negative (TN), and
>> false_negative (FN).
>>
>> I am not sure the true_positive value is for class a or class b in the
>> Experimenter output.
>> I think the true_positive value for class a is referred as the
>> true_negative for class b.
>>
>> Can someone kindly let us know the definition when you can get a chance.
>>
>> Thanks,
>> Shu-Ju
>> _______________________________________________
>> Wekalist mailing list
>> Send posts to: [hidden email]
>> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
>> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: True_positive/True_negative definition in the output report of Experimenter

Eibe Frank-2
Administrator

> On 25/05/2017, at 8:35 PM, Shu-Ju Tu <[hidden email]> wrote:
>
> What is the exact meaning for the first class as the positive class?
>
> Does that mean the class assigned to the very first instance?
>
> Or the class "a" as appeared in the output of Explorer?

The first class value in the list of class attribute values in the corresponding ARFF file. This is the first value that is shown in the Preprocess panel of the Explorer.

If you load a CSV file into WEKA, it will be the first class value that occurs in the file (unless you explicitly provide the list of attribute values as a parameter for the CSV loader).

> If I want to double check it with Explorer/Experimenter, will the
> result of Experimenter be the same as in the output of Explorer if we
> set the repetition to 1?

Yes, if you use cross-validation as the evaluation option.

Cheers,
Eibe
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Loading...