Outliers vs extreme values

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Outliers vs extreme values

asadbtk
Hello

In general data science, outliers are extreme values but in weka they are given separate, and we can exclude both outliers and extreme values from a dataset. What is the difference between these two? 

Best regards 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Outliers vs extreme values

Peter Reutemann
> In general data science, outliers are extreme values but in weka they are given separate, and we can exclude both outliers and extreme values from a dataset. What is the difference between these two?

I presume you're referring to the InterquartileRange filter. Extreme
values are also outliers, just extreme ones. The notion of extreme
values gives you the ability to divide outliers into two bands, where
outliers happen every now an then, but extreme values only very
rarely. In terms of analyzing samples with an instrument (eg near
infrared instrument), you can think of outliers as wrong type of
sample being analyzed (= human error, but still correct data returned
from the sensor) and extreme values as sensor saturation (completely
unexpected response from the sensor). What values to chose in order to
distinguish between outlier/extreme value will be domain dependent.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Outliers vs extreme values

asadbtk
Thanks a lot Peter. 

Best regards 

On Wednesday, January 8, 2020, Peter Reutemann <[hidden email]> wrote:
> In general data science, outliers are extreme values but in weka they are given separate, and we can exclude both outliers and extreme values from a dataset. What is the difference between these two?

I presume you're referring to the InterquartileRange filter. Extreme
values are also outliers, just extreme ones. The notion of extreme
values gives you the ability to divide outliers into two bands, where
outliers happen every now an then, but extreme values only very
rarely. In terms of analyzing samples with an instrument (eg near
infrared instrument), you can think of outliers as wrong type of
sample being analyzed (= human error, but still correct data returned
from the sensor) and extreme values as sensor saturation (completely
unexpected response from the sensor). What values to chose in order to
distinguish between outlier/extreme value will be domain dependent.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html