JRip Confusion

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

JRip Confusion

Fer137
I do not understand this result:
CostSensitiveClassifier with JRip classifying between "A" and "B". This
usually gives a few jrip-rules, but sometimes gives only 1 "all B" rule.
Interestingly, in this example, the Confusion Matrix gives several "A"
results. How is it possible if there is only one rule that would always give
"B"?
What rule makes any "A" appear?

======================================================
weka.classifiers.meta.CostSensitiveClassifier -cost-matrix "[0.0 5.0; 15.0
0.0]" -S 1 -W weka.classifiers.rules.JRip -- -F 3 -N 2.0 -O 2 -S 1
...
JRIP rules:
===========
 => Clase=B (121776.99999982314/9590.326032339784)
Number of Rules : 1

=== Confusion Matrix ===
     a     b   <-- classified as
    12 10049 |     a = A
     8 38642 |     b = B



--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: JRip Confusion

Peter Reutemann-3
On January 15, 2020 3:28:05 AM GMT+13:00, Fer137 <[hidden email]> wrote:

>I do not understand this result:
>CostSensitiveClassifier with JRip classifying between "A" and "B". This
>usually gives a few jrip-rules, but sometimes gives only 1 "all B"
>rule.
>Interestingly, in this example, the Confusion Matrix gives several "A"
>results. How is it possible if there is only one rule that would always
>give
>"B"?
>What rule makes any "A" appear?
>
>======================================================
>weka.classifiers.meta.CostSensitiveClassifier -cost-matrix "[0.0 5.0;
>15.0
>0.0]" -S 1 -W weka.classifiers.rules.JRip -- -F 3 -N 2.0 -O 2 -S 1
>...
>JRIP rules:
>===========
> => Clase=B (121776.99999982314/9590.326032339784)
>Number of Rules : 1
>
>=== Confusion Matrix ===
>     a     b   <-- classified as
>    12 10049 |     a = A
>     8 38642 |     b = B
>
>
>
>--
>Sent from: https://weka.8497.n7.nabble.com/
>_______________________________________________
>Wekalist mailing list -- [hidden email]
>Send posts to: To unsubscribe send an email to
>[hidden email]
>To subscribe, unsubscribe, etc., visit
>https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
>List etiquette:
>http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

Did you run a cross-validation (CV)?

The model that gets output in such a case, was built on the *full* dataset.

During the evaluation of the 10 models of your 10-fold CV, some models may jave resulted in different rules, generating the confusion matrix that you posted.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: JRip Confusion

Fer137
No. I always do it with %Split and order preserved, or with a different test
file. In this case with the first. (Precisely to avoid weird things of cross
validation)
Hence my surprise with that result.

Regards.




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: JRip Confusion

Eibe Frank-3
The same applies when you use a percentage split (regardless of whether you preserve the order): the model that is output in the Classify tab is the one built from the *full* dataset as loaded into the Preprocess tab.

In the latest versions of WEKA, you can actually see all the models built from subsets of the data during evaluation by choosing the appropriate tick box under "More options..." in the Classify tab.

Cheers,
Eibe

On Thu, Jan 16, 2020 at 12:22 AM Fer137 <[hidden email]> wrote:
No. I always do it with %Split and order preserved, or with a different test
file. In this case with the first. (Precisely to avoid weird things of cross
validation)
Hence my surprise with that result.

Regards.




--
Sent from: https://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to: To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit
https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html