evaluation metrics

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

evaluation metrics

Bhupesh Rawat
Dear Sir,

İ m using RMSE and precision to evaluate recommendations. İ'd like to know the relationship between these two. İs it possible to have nearly similar values of these two.

Thanx a lot.





hanx

--
Thanks & Regards
Bhupesh Rawat.
Ph.D Scholar
Department of Computer Science,Babasaheb Bhimrao Ambedkar University
Vidya Vihar,Rai Bareilly road(Lucknow)
Ph. No: +91-9897065948

...........................................................................................................................
*A man is the best judge of himself and he has to pay the price for what he
does.*
...........................................................................................................................





_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: evaluation metrics

Eibe Frank-3
There is no strong relationship. Consider a two-class case where all positive instances get probability P(pos) = 0.51 and all negative instances get probability P(pos) = 0.49. Precision will be 1 with the default classification threshold of 0.5. However, RMSE (i.e., the square root of the "original" Brier score) will be terrible (close to 0.5).

Cheers,
Eibe

On Tue, Feb 20, 2018 at 5:52 PM, Bhupesh Rawat <[hidden email]> wrote:
Dear Sir,

İ m using RMSE and precision to evaluate recommendations. İ'd like to know the relationship between these two. İs it possible to have nearly similar values of these two.

Thanx a lot.





hanx

--
Thanks & Regards
Bhupesh Rawat.
Ph.D Scholar
Department of Computer Science,Babasaheb Bhimrao Ambedkar University
Vidya Vihar,Rai Bareilly road(Lucknow)
Ph. No: <a href="tel:+91%2098970%2065948" value="+919897065948" target="_blank">+91-9897065948

...........................................................................................................................
*A man is the best judge of himself and he has to pay the price for what he
does.*
...........................................................................................................................





_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html



_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: evaluation metrics

Bhupesh Rawat
Dear Sir,

How do i explain the results in the attached file particularly in the
third row where the difference between RMSE and precision is small.

The accuracy of course recommended is shown for each cluster of learner.


On 2/20/18, Eibe Frank <[hidden email]> wrote:

> There is no strong relationship. Consider a two-class case where all
> positive instances get probability P(pos) = 0.51 and all negative instances
> get probability P(pos) = 0.49. Precision will be 1 with the default
> classification threshold of 0.5. However, RMSE (i.e., the square root of
> the "original" Brier score) will be terrible (close to 0.5).
>
> Cheers,
> Eibe
>
> On Tue, Feb 20, 2018 at 5:52 PM, Bhupesh Rawat <[hidden email]> wrote:
>
>> Dear Sir,
>>
>> İ m using RMSE and precision to evaluate recommendations. İ'd like to
>> know
>> the relationship between these two. İs it possible to have nearly similar
>> values of these two.
>>
>> Thanx a lot.
>>
>>
>>
>>
>>
>> hanx
>>
>> --
>> Thanks & Regards
>> Bhupesh Rawat.
>> Ph.D Scholar
>> Department of Computer Science,Babasaheb Bhimrao Ambedkar University
>> Vidya Vihar,Rai Bareilly road(Lucknow)
>> Ph. No: +91-9897065948 <+91%2098970%2065948>
>>
>> ............................................................
>> ...............................................................
>> *A man is the best judge of himself and he has to pay the price for what
>> he
>> does.*
>> ............................................................
>> ...............................................................
>>
>>
>>
>>
>>
>> _______________________________________________
>> Wekalist mailing list
>> Send posts to: [hidden email]
>> List info and subscription status: https://list.waikato.ac.nz/
>> mailman/listinfo/wekalist
>> List etiquette: http://www.cs.waikato.ac.nz/~
>> ml/weka/mailinglist_etiquette.html
>>
>>
>

--
Thanks & Regards
Bhupesh Rawat.
Ph.D Scholar
Department of Computer Science,Babasaheb Bhimrao Ambedkar University
Vidya Vihar,Rai Bareilly road(Lucknow)
Ph. No: +91-9897065948

...........................................................................................................................
*A man is the best judge of himself and he has to pay the price for what he
does.*
...........................................................................................................................

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

Table12.tif (1006K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: evaluation metrics

Bhupesh Rawat
In reply to this post by Eibe Frank-3
Dear Sir,

Could you clarify the example Further. How the RMSE of 'original brier
score ' is close to 0.5.

On 2/20/18, Eibe Frank <[hidden email]> wrote:

> There is no strong relationship. Consider a two-class case where all
> positive instances get probability P(pos) = 0.51 and all negative instances
> get probability P(pos) = 0.49. Precision will be 1 with the default
> classification threshold of 0.5. However, RMSE (i.e., the square root of
> the "original" Brier score) will be terrible (close to 0.5).
>
> Cheers,
> Eibe
>
> On Tue, Feb 20, 2018 at 5:52 PM, Bhupesh Rawat <[hidden email]> wrote:
>
>> Dear Sir,
>>
>> İ m using RMSE and precision to evaluate recommendations. İ'd like to
>> know
>> the relationship between these two. İs it possible to have nearly similar
>> values of these two.
>>
>> Thanx a lot.
>>
>>
>>
>>
>>
>> hanx
>>
>> --
>> Thanks & Regards
>> Bhupesh Rawat.
>> Ph.D Scholar
>> Department of Computer Science,Babasaheb Bhimrao Ambedkar University
>> Vidya Vihar,Rai Bareilly road(Lucknow)
>> Ph. No: +91-9897065948 <+91%2098970%2065948>
>>
>> ............................................................
>> ...............................................................
>> *A man is the best judge of himself and he has to pay the price for what
>> he
>> does.*
>> ............................................................
>> ...............................................................
>>
>>
>>
>>
>>
>> _______________________________________________
>> Wekalist mailing list
>> Send posts to: [hidden email]
>> List info and subscription status: https://list.waikato.ac.nz/
>> mailman/listinfo/wekalist
>> List etiquette: http://www.cs.waikato.ac.nz/~
>> ml/weka/mailinglist_etiquette.html
>>
>>
>


--
Thanks & Regards
Bhupesh Rawat.
Ph.D Scholar
Department of Computer Science,Babasaheb Bhimrao Ambedkar University
Vidya Vihar,Rai Bareilly road(Lucknow)
Ph. No: +91-9897065948

...........................................................................................................................
*A man is the best judge of himself and he has to pay the price for what he
does.*
...........................................................................................................................
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: evaluation metrics

Eibe Frank-3
I never said anything about "RMSE of original Brier score". The RMSE output by WEKA *is* the "original" Brier score. Take a look at the Wikipedia article on the Brier score.

Cheers,
Eibe

On Thu, Feb 22, 2018 at 9:10 PM, Bhupesh Rawat <[hidden email]> wrote:
Dear Sir,

Could you clarify the example Further. How the RMSE of 'original brier
score ' is close to 0.5.

On 2/20/18, Eibe Frank <[hidden email]> wrote:
> There is no strong relationship. Consider a two-class case where all
> positive instances get probability P(pos) = 0.51 and all negative instances
> get probability P(pos) = 0.49. Precision will be 1 with the default
> classification threshold of 0.5. However, RMSE (i.e., the square root of
> the "original" Brier score) will be terrible (close to 0.5).
>
> Cheers,
> Eibe
>
> On Tue, Feb 20, 2018 at 5:52 PM, Bhupesh Rawat <[hidden email]> wrote:
>
>> Dear Sir,
>>
>> İ m using RMSE and precision to evaluate recommendations. İ'd like to
>> know
>> the relationship between these two. İs it possible to have nearly similar
>> values of these two.
>>
>> Thanx a lot.
>>
>>
>>
>>
>>
>> hanx
>>
>> --
>> Thanks & Regards
>> Bhupesh Rawat.
>> Ph.D Scholar
>> Department of Computer Science,Babasaheb Bhimrao Ambedkar University
>> Vidya Vihar,Rai Bareilly road(Lucknow)
>> Ph. No: <a href="tel:%2B91-9897065948" value="+919897065948">+91-9897065948 <+91%2098970%2065948>
>>
>> ............................................................
>> ...............................................................
>> *A man is the best judge of himself and he has to pay the price for what
>> he
>> does.*
>> ............................................................
>> ...............................................................
>>
>>
>>
>>
>>
>> _______________________________________________
>> Wekalist mailing list
>> Send posts to: [hidden email]
>> List info and subscription status: https://list.waikato.ac.nz/
>> mailman/listinfo/wekalist
>> List etiquette: http://www.cs.waikato.ac.nz/~
>> ml/weka/mailinglist_etiquette.html
>>
>>
>


--
Thanks & Regards
Bhupesh Rawat.
Ph.D Scholar
Department of Computer Science,Babasaheb Bhimrao Ambedkar University
Vidya Vihar,Rai Bareilly road(Lucknow)
Ph. No: <a href="tel:%2B91-9897065948" value="+919897065948">+91-9897065948

...........................................................................................................................
*A man is the best judge of himself and he has to pay the price for what he
does.*
...........................................................................................................................
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html