Role of classifier in Feature selection

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Role of classifier in Feature selection

neha.bologna
Hi Eibe and Peter

I am using AttributeSelectedClassifier for feature selection. I just want to see the consistency of feature selection algorithms across different folds of k-fold CV and have no interest to evaluate the prediction accuracy based on the selected features. 

My question: is there still the role of classifiers (i.e. RF) in feature selection? In other words, if I select RF as a classifier and select features and then I use SVM and select features, can I expect different subset of features selected?  Do feature selection algorithms use RMSE etc as a fitness function to select the features? 

Thank you in advance. 

Regards

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Role of classifier in Feature selection

Peter Reutemann
> I am using AttributeSelectedClassifier for feature selection. I just want to see the consistency of feature selection algorithms across different folds of k-fold CV and have no interest to evaluate the prediction accuracy based on the selected features.
>
> My question: is there still the role of classifiers (i.e. RF) in feature selection? In other words, if I select RF as a classifier and select features and then I use SVM and select features, can I expect different subset of features selected?  Do feature selection algorithms use RMSE etc as a fitness function to select the features?

The AttributeSelectedClassifier meta-classifier simply trains on the
reduced dataset that the specified feature selection pair of search
and evaluation generate (the feature selection is a preprocessing step
within the meta-classifier). The specified classifier that gets
trained has no influence on the feature selection process.
If you select ClassifierSubsetEval as evaluation method then the
classifier selected within that evaluation method has an impact on the
features that get selected (but not the one that you specified within
AttributeSelectedClassifier).

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Role of classifier in Feature selection

neha.bologna
Hello Peter

if you select ClassifierSubsetEval as evaluation method then the
classifier selected within that evaluation method has an impact on the
features that get selected  

But the  ClassifierSubsetEval is inside the AttributeSelectedClassifier under the option 'evaluator' ?



On Thu, Oct 29, 2020 at 2:40 AM Peter Reutemann <[hidden email]> wrote:
> I am using AttributeSelectedClassifier for feature selection. I just want to see the consistency of feature selection algorithms across different folds of k-fold CV and have no interest to evaluate the prediction accuracy based on the selected features.
>
> My question: is there still the role of classifiers (i.e. RF) in feature selection? In other words, if I select RF as a classifier and select features and then I use SVM and select features, can I expect different subset of features selected?  Do feature selection algorithms use RMSE etc as a fitness function to select the features?

The AttributeSelectedClassifier meta-classifier simply trains on the
reduced dataset that the specified feature selection pair of search
and evaluation generate (the feature selection is a preprocessing step
within the meta-classifier). The specified classifier that gets
trained has no influence on the feature selection process.
If you select ClassifierSubsetEval as evaluation method then the
classifier selected within that evaluation method has an impact on the
features that get selected (but not the one that you specified within
AttributeSelectedClassifier).

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Role of classifier in Feature selection

Peter Reutemann
> if you select ClassifierSubsetEval as evaluation method then the
> classifier selected within that evaluation method has an impact on the
> features that get selected
>
> But the  ClassifierSubsetEval is inside the AttributeSelectedClassifier under the option 'evaluator' ?

I'm not sure what your point is. The inner classifier influences the
features that the outer classifier then will receive as training data.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 577-5304
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html