How to execute from the command-line an AutoWEKA result?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to execute from the command-line an AutoWEKA result?

Gabriel  Del Rio
Hi,

After successfully running auto-weka, I obtained optimum algorithm and hyper parameters for a given dataset. For instance, I got:

best classifier: weka.classifiers.lazy.LWL
arguments: [-A, weka.core.neighboursearch.LinearNNSearch, -W, weka.classifiers.functions.MultilayerPerceptron, --, -L, 0.3899191912662868, -M, 0.4683563849238558, -H, o, -C, -R, -D, -S, 1]
attribute search: null
attribute search arguments: []
attribute evaluation: null
attribute evaluation arguments: []
estimated error: 15.87743732590529

Then, I tried to execute this algorithm with its parameters from the command line as follows:

java -Xmx32g weka.classifiers.lazy.LWL -A "weka.core.neighboursearch.LinearNNSearch" -W "weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1" -t myarffile.arff

(Note, in my computer I have previously set the CLASSPATH so I do not need to call the jar files from weka)

This command renders and error:

Weka exception: weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1

General options:

-h or -help
Output help information.
-synopsis or -info
…. here goes the options to execute weka MultilayerPerceptron

If I change the MultilayerPerceptron parameters to "-- -L 0.3899191912662868 -M 0.4683563849238558 -N 500 -V 0 -E 20 -S 1 -H a", then I got the following error:

Weka exception: weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -N 500 -V 0 -E 20 -S 1 -H a

General options:

-h or -help
Output help information.
-synopsis or -info
here goes the options to execute weka MultilayerPerceptron

I have changed the command-line using different parameters that are part of the MultilayerPerceptron (for example, get rind of the "--", etc), and none worked so far, so I'm not sure what I'm doing wrong...Has anyone had tried to execute previously auto-weka best classifier with the specified parameters? Any suggestion on how to succeed in executing the resulting algorithm-parameters?

Any help is appreciated :)

Gabriel




_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to execute from the command-line an AutoWEKA result?

Eibe Frank-2
Administrator

> On 21 May 2017, at 07:08, Gabriel Del Rio <[hidden email]> wrote:
>
> java -Xmx32g weka.classifiers.lazy.LWL -A "weka.core.neighboursearch.LinearNNSearch" -W "weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1" -t myarffile.arff

The double-hyphen (--) is used to separate parameters for the base learner given by -W from the parameters for the first-level classifier (in your case, LWL). No quotes are necessary when specifying parameters for the base learner. General options for evaluating a classifier, e.g., specifying the training set, must always occur before the first double-hyphen.

Try

java -Xmx32g weka.classifiers.lazy.LWL -t myarfffile.arff -A weka.core.neighboursearch.LinearNNSearch -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1

Cheers,
Eibe
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to execute from the command-line an AutoWEKA result?

Gabriel  Del Rio
Dear Eibe,

Thanks, that solved the problem. I have now another problem :(, which is that I can not reproduce the reported efficiencies in the prediction reported by AutoWEKA using the parameters and the algorithm… For instance, for a given arff file, AutoWEKA reports the following:

best classifier: weka.classifiers.bayes.NaiveBayes
arguments: [-D]
attribute search: weka.attributeSelection.GreedyStepwise
attribute search arguments: [-C, -N, 161]
attribute evaluation: weka.attributeSelection.CfsSubsetEval
attribute evaluation arguments: [-L]
estimated error: 19.191919191919197


Correctly Classified Instances         560               80.8081 %
Incorrectly Classified Instances       133               19.1919 %


Using the following command line to reproduce autoWEKA results:

java -Xmx32g weka.classifiers.bayes.NaiveBayes -t arffile.arff -D

I obtained the following results:

=== Error on training data ===

Correctly Classified Instances         489               70.5628 %
Incorrectly Classified Instances       204               29.4372 %

What am I missing here?

Best,

Gabriel




From: "Eibe Frank" <[hidden email]>
To: "Weka machine learning workbench list." <[hidden email]>
Sent: Saturday, May 20, 2017 8:23:24 PM
Subject: Re: [Wekalist] How to execute from the command-line an AutoWEKA        result?


> On 21 May 2017, at 07:08, Gabriel Del Rio <[hidden email]> wrote:
>
> java -Xmx32g weka.classifiers.lazy.LWL -A "weka.core.neighboursearch.LinearNNSearch" -W "weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1" -t myarffile.arff

The double-hyphen (--) is used to separate parameters for the base learner given by -W from the parameters for the first-level classifier (in your case, LWL). No quotes are necessary when specifying parameters for the base learner. General options for evaluating a classifier, e.g., specifying the training set, must always occur before the first double-hyphen.

Try

java -Xmx32g weka.classifiers.lazy.LWL -t myarfffile.arff -A weka.core.neighboursearch.LinearNNSearch -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1

Cheers,
Eibe
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to execute from the command-line an AutoWEKA result?

Eibe Frank-2
Administrator
You also need to include attribute selection in your command-line. The specification in your output would correspond to

java weka.Run .AttributeSelectedClassifier -t ~/datasets/UCI/iris.arff -S ".GreedyStepwise -C -N 161" -E ".CfsSubsetEval -L" -W .NaiveBayes -- -D

where I’ve just used the iris data as an example.

Cheers,
Eibe

> On 23/05/2017, at 12:00 PM, Gabriel Del Rio <[hidden email]> wrote:
>
> Dear Eibe,
>
> Thanks, that solved the problem. I have now another problem :(, which is that I can not reproduce the reported efficiencies in the prediction reported by AutoWEKA using the parameters and the algorithm… For instance, for a given arff file, AutoWEKA reports the following:
>
> best classifier: weka.classifiers.bayes.NaiveBayes
> arguments: [-D]
> attribute search: weka.attributeSelection.GreedyStepwise
> attribute search arguments: [-C, -N, 161]
> attribute evaluation: weka.attributeSelection.CfsSubsetEval
> attribute evaluation arguments: [-L]
> estimated error: 19.191919191919197
>
>
> Correctly Classified Instances         560               80.8081 %
> Incorrectly Classified Instances       133               19.1919 %
>
>
> Using the following command line to reproduce autoWEKA results:
>
> java -Xmx32g weka.classifiers.bayes.NaiveBayes -t arffile.arff -D
>
> I obtained the following results:
>
> === Error on training data ===
>
> Correctly Classified Instances         489               70.5628 %
> Incorrectly Classified Instances       204               29.4372 %
>
> What am I missing here?
>
> Best,
>
> Gabriel
>
>
>
> From: "Eibe Frank" <[hidden email]>
> To: "Weka machine learning workbench list." <[hidden email]>
> Sent: Saturday, May 20, 2017 8:23:24 PM
> Subject: Re: [Wekalist] How to execute from the command-line an AutoWEKA        result?
>
>
> > On 21 May 2017, at 07:08, Gabriel Del Rio <[hidden email]> wrote:
> >
> > java -Xmx32g weka.classifiers.lazy.LWL -A "weka.core.neighboursearch.LinearNNSearch" -W "weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1" -t myarffile.arff
>
> The double-hyphen (--) is used to separate parameters for the base learner given by -W from the parameters for the first-level classifier (in your case, LWL). No quotes are necessary when specifying parameters for the base learner. General options for evaluating a classifier, e.g., specifying the training set, must always occur before the first double-hyphen.
>
> Try
>
> java -Xmx32g weka.classifiers.lazy.LWL -t myarfffile.arff -A weka.core.neighboursearch.LinearNNSearch -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1
>
> Cheers,
> Eibe
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to execute from the command-line an AutoWEKA result?

Gabriel  Del Rio
Thank you again Eibe, that solved the problem. 

Best,

Gabriel


From: "Eibe Frank" <[hidden email]>
To: "Weka machine learning workbench list." <[hidden email]>
Sent: Monday, May 22, 2017 8:08:34 PM
Subject: Re: [Wekalist] How to execute from the command-line an AutoWEKA        result?

You also need to include attribute selection in your command-line. The specification in your output would correspond to

java weka.Run .AttributeSelectedClassifier -t ~/datasets/UCI/iris.arff -S ".GreedyStepwise -C -N 161" -E ".CfsSubsetEval -L" -W .NaiveBayes -- -D

where I’ve just used the iris data as an example.

Cheers,
Eibe

> On 23/05/2017, at 12:00 PM, Gabriel Del Rio <[hidden email]> wrote:
>
> Dear Eibe,
>
> Thanks, that solved the problem. I have now another problem :(, which is that I can not reproduce the reported efficiencies in the prediction reported by AutoWEKA using the parameters and the algorithm… For instance, for a given arff file, AutoWEKA reports the following:
>
> best classifier: weka.classifiers.bayes.NaiveBayes
> arguments: [-D]
> attribute search: weka.attributeSelection.GreedyStepwise
> attribute search arguments: [-C, -N, 161]
> attribute evaluation: weka.attributeSelection.CfsSubsetEval
> attribute evaluation arguments: [-L]
> estimated error: 19.191919191919197
>
>
> Correctly Classified Instances         560               80.8081 %
> Incorrectly Classified Instances       133               19.1919 %
>
>
> Using the following command line to reproduce autoWEKA results:
>
> java -Xmx32g weka.classifiers.bayes.NaiveBayes -t arffile.arff -D
>
> I obtained the following results:
>
> === Error on training data ===
>
> Correctly Classified Instances         489               70.5628 %
> Incorrectly Classified Instances       204               29.4372 %
>
> What am I missing here?
>
> Best,
>
> Gabriel
>
>
>
> From: "Eibe Frank" <[hidden email]>
> To: "Weka machine learning workbench list." <[hidden email]>
> Sent: Saturday, May 20, 2017 8:23:24 PM
> Subject: Re: [Wekalist] How to execute from the command-line an AutoWEKA        result?
>
>
> > On 21 May 2017, at 07:08, Gabriel Del Rio <[hidden email]> wrote:
> >
> > java -Xmx32g weka.classifiers.lazy.LWL -A "weka.core.neighboursearch.LinearNNSearch" -W "weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1" -t myarffile.arff
>
> The double-hyphen (--) is used to separate parameters for the base learner given by -W from the parameters for the first-level classifier (in your case, LWL). No quotes are necessary when specifying parameters for the base learner. General options for evaluating a classifier, e.g., specifying the training set, must always occur before the first double-hyphen.
>
> Try
>
> java -Xmx32g weka.classifiers.lazy.LWL -t myarfffile.arff -A weka.core.neighboursearch.LinearNNSearch -W weka.classifiers.functions.MultilayerPerceptron -- -L 0.3899191912662868 -M 0.4683563849238558 -H o -C -R -D -S 1
>
> Cheers,
> Eibe
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html