Simulating AutoWeka manually...

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Simulating AutoWeka manually...

erkanozhan
Hi,

I analyzed the data set with AutoWeka for 800 minutes, 5 threads. The result was very interesting. You can see these results below.


"Auto-WEKA result:
best classifier: weka.classifiers.meta.AttributeSelectedClassifier
arguments: [-S, weka.attributeSelection.GreedyStepwise, -E, weka.attributeSelection.CfsSubsetEval, -W, weka.classifiers.trees.RandomForest, --, -I, 76, -K, 1, -depth, 13]
attribute search: null
attribute search arguments: []
attribute evaluation: null
attribute evaluation arguments: []
metric: errorRate
estimated errorRate: -1.0
training time on evaluation dataset: 0.047 seconds

You can use the chosen classifier in your own code as follows:


Classifier classifier = AbstractClassifier.forName("weka.classifiers.meta.AttributeSelectedClassifier", new String[]{"-S", "weka.attributeSelection.GreedyStepwise", "-E", "weka.attributeSelection.CfsSubsetEval", "-W", "weka.classifiers.trees.RandomForest", "--", "-I", "76", "-K", "1", "-depth", "13"});
classifier.buildClassifier(instances);



Correctly Classified Instances         106              100      %
Incorrectly Classified Instances         0                0      %
Kappa statistic                          1    
Mean absolute error                      0.0747
Root mean squared error                  0.0986
Relative absolute error                 14.94   %
Root relative squared error             19.7174 %
Total Number of Instances              106    

=== Confusion Matrix ===

  a  b   <-- classified as
 53  0 |  a = +
  0 53 |  b = -

=== Detailed Accuracy By Class ===

                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC      ROC Area  PRC Area  Class
                 1,000    0,000    1,000      1,000    1,000      1,000    1,000     1,000     +
                 1,000    0,000    1,000      1,000    1,000      1,000    1,000     1,000     -
Weighted Avg.    1,000    0,000    1,000      1,000    1,000      1,000    1,000     1,000     "

"<a href="https://www.cs.ubc.ca/labs/beta/Projects/autoweka/manual.pdf">https://www.cs.ubc.ca/labs/beta/Projects/autoweka/manual.pdf</a>" I could not solve it here.




1- My question is: How can I provide the parameters here manually.
I went to the "SelectAttributes" section in Weka. According to AutoWeka, the parameters are as follows:

"arguments: [-S, weka.attributeSelection.GreedyStepwise, -E, weka.attributeSelection.CfsSubsetEval, -W, weka.classifiers.trees.RandomForest, -, -I, 76, -K, 1, -depth, 13 ] ".

Whatever I did here, I couldn't set the "-S, -E and -W" parameters. How can I do that?


bolmeler.png


It is also necessary to set these parameters for RandomForest.

2- Another problem is that after running the AttributeSelector, it gets all the data when you issue the "Save Reduced Data" command. It replaces unnecessary data with 0 and you need to clean it manually.
2-1- How will I see what the selected attributes are. Are these proportions? I think it's not on GreedyStepwise. I guess it happens when you choose Ranker.

3- The latest 95% success has been achieved with the data set I have worked with. Does my result mean progress?

I apologize for writing long. I wish you healthy and happy days.

Cheers, 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Simulating AutoWeka manually...

Peter Reutemann-3
For question 1:
AutoWeka outputs the options for the meta classifier weka.classifiers.meta.AttributeSelectedClassifier. Why are you not using this in the classify tab?
If you really want to use the attribute selection tab, then first copy that setup into the classify tab and then copy/paste the search and evaluation methods individually across into the attribute selection tab.

Cheers, Peter

On August 23, 2020 8:00:12 AM GMT+12:00, "Erkan Özhan" <[hidden email]> wrote:

>Hi,
>
>I analyzed the data set with AutoWeka for 800 minutes, 5 threads. The
>result was very interesting. You can see these results below.
>
>
>"Auto-WEKA result:
>best classifier: weka.classifiers.meta.AttributeSelectedClassifier
>arguments: [-S, weka.attributeSelection.GreedyStepwise, -E,
>weka.attributeSelection.CfsSubsetEval, -W,
>weka.classifiers.trees.RandomForest, --, -I, 76, -K, 1, -depth, 13]
>attribute search: null
>attribute search arguments: []
>attribute evaluation: null
>attribute evaluation arguments: []
>metric: errorRate
>estimated errorRate: -1.0
>training time on evaluation dataset: 0.047 seconds
>
>You can use the chosen classifier in your own code as follows:
>
>
>Classifier classifier =
>AbstractClassifier.forName("weka.classifiers.meta.AttributeSelectedClassifier",
>new String[]{"-S", "weka.attributeSelection.GreedyStepwise", "-E",
>"weka.attributeSelection.CfsSubsetEval", "-W",
>"weka.classifiers.trees.RandomForest", "--", "-I", "76", "-K", "1",
>"-depth", "13"});
>classifier.buildClassifier(instances);
>
>
>
>Correctly Classified Instances         106              100      %
>Incorrectly Classified Instances         0                0      %
>Kappa statistic                          1
>Mean absolute error                      0.0747
>Root mean squared error                  0.0986
>Relative absolute error                 14.94   %
>Root relative squared error             19.7174 %
>Total Number of Instances              106
>
>=== Confusion Matrix ===
>
>  a  b   <-- classified as
> 53  0 |  a = +
>  0 53 |  b = -
>
>=== Detailed Accuracy By Class ===
>
>                 TP Rate  FP Rate  Precision  Recall   F-Measure  MCC
> ROC Area  PRC Area  Class
>                 1,000    0,000    1,000      1,000    1,000      1,000
> 1,000     1,000     +
>                 1,000    0,000    1,000      1,000    1,000      1,000
> 1,000     1,000     -
>Weighted Avg.    1,000    0,000    1,000      1,000    1,000      1,000
> 1,000     1,000     "
>
>"<a
>href="https://www.cs.ubc.ca/labs/beta/Projects/autoweka/manual.pdf">
>https://www.cs.ubc.ca/labs/beta/Projects/autoweka/manual.pdf</a>" I
>could
>not solve it here.
>
>
>
>
>1- My question is: How can I provide the parameters here manually.
>I went to the "SelectAttributes" section in Weka. According to
>AutoWeka,
>the parameters are as follows:
>
>"arguments: [-S, weka.attributeSelection.GreedyStepwise, -E,
>weka.attributeSelection.CfsSubsetEval, -W,
>weka.classifiers.trees.RandomForest, -, -I, 76, -K, 1, -depth, 13 ] ".
>
>Whatever I did here, I couldn't set the "-S, -E and -W" parameters. How
>can
>I do that?
>
>
>[image: bolmeler.png]
>
>
>It is also necessary to set these parameters for RandomForest.
>
>2- Another problem is that after running the AttributeSelector, it gets
>all
>the data when you issue the "Save Reduced Data" command. It replaces
>unnecessary data with 0 and you need to clean it manually.
>2-1- How will I see what the selected attributes are. Are these
>proportions? I think it's not on GreedyStepwise. I guess it happens
>when
>you choose Ranker.
>
>3- The latest 95% success has been achieved with the data set I have
>worked
>with. Does my result mean progress?
>
>I apologize for writing long. I wish you healthy and happy days.
>
>Cheers,

--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html