How to select different samples for a FS algorithm

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to select different samples for a FS algorithm

neha.bologna
Hello to everyone

I have a question. If I have to use a particular feature selection algorithm (i.e. Genetic Search) and apply to different datasets in Weka explorer. How can I change the different training samples in order to evaluate if Genetic Search selects same features of a dataset when we change the training sample?

Do a change in training sample means using different fold in k-fold CV? I mean using first 10 fold CV and then 5 fold CV or it means excluding some of the instances from training data?

Thanks

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to select different samples for a FS algorithm

Peter Reutemann
> I have a question. If I have to use a particular feature selection algorithm (i.e. Genetic Search) and apply to different datasets in Weka explorer. How can I change the different training samples in order to evaluate if Genetic Search selects same features of a dataset when we change the training sample?
>
> Do a change in training sample means using different fold in k-fold CV? I mean using first 10 fold CV and then 5 fold CV or it means excluding some of the instances from training data?

If you want to keep the same K for your CV, you can change the seed
value for the randomization.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to select different samples for a FS algorithm

neha.bologna
Thanks for your reply Peter. 

If we have already selected a seed value of one, then what would be the difference if we select two or select a seed value of 10? 

On Sunday, June 28, 2020, Peter Reutemann <[hidden email]> wrote:
> I have a question. If I have to use a particular feature selection algorithm (i.e. Genetic Search) and apply to different datasets in Weka explorer. How can I change the different training samples in order to evaluate if Genetic Search selects same features of a dataset when we change the training sample?
>
> Do a change in training sample means using different fold in k-fold CV? I mean using first 10 fold CV and then 5 fold CV or it means excluding some of the instances from training data?

If you want to keep the same K for your CV, you can change the seed
value for the randomization.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to select different samples for a FS algorithm

Peter Reutemann
> If we have already selected a seed value of one, then what would be the difference if we select two or select a seed value of 10?

Since it is randomization, I can't really tell you. ;-)

Any cross-validation run shuffles the data before splitting it into
folds. If you want to see how the data gets randomized, you can use
the Randomize filter (package weka.filters.unsupervised.instance) and
set the specific seed value.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: How to select different samples for a FS algorithm

neha.bologna
OK thank you Peter for your time. 



On Monday, June 29, 2020, Peter Reutemann <[hidden email]> wrote:
> If we have already selected a seed value of one, then what would be the difference if we select two or select a seed value of 10?

Since it is randomization, I can't really tell you. ;-)

Any cross-validation run shuffles the data before splitting it into
folds. If you want to see how the data gets randomized, you can use
the Randomize filter (package weka.filters.unsupervised.instance) and
set the specific seed value.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html