Regression - supervised learning

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Regression - supervised learning

Alexander Osherenko
Hi!

Just an idea. I know it is feasible, but does it actually make sense?

I am thinking about regression and supervised learning. Regression models don't work with nominal or string values but there is a workaround to avoid corresponding problems. For example, can I transform the supervised outcome of an experiment from a nominal/string in a numeric value, for instance, using the StringToNominal/NominalToBinary filters, train a regression model using the corresponding dataset, evaluate an outcome as a numeric value and transfer this outcome back in the nominal value?

The advantage: the outcome can be nominal or string and I don't have to specify it beforehand since everything is dynamic.

Best, Alexander

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Eibe Frank-2
Administrator
There is a classifier in WEKA, ClassificationViaRegression, that can be used to apply regression schemes to classification problems with a nominal class attribute.

Your idea regarding string attributes is interesting. I suppose you would get multiple target attributes this way, so you would need to apply multi-target regression. WEKA doesn’t support this (although you could, programmatically, via the WEKA API, build a separate model for each target attribute).

Cheers,
Eibe

> On 12/05/2017, at 1:06 AM, Alexander Osherenko <[hidden email]> wrote:
>
> Hi!
>
> Just an idea. I know it is feasible, but does it actually make sense?
>
> I am thinking about regression and supervised learning. Regression models don't work with nominal or string values but there is a workaround to avoid corresponding problems. For example, can I transform the supervised outcome of an experiment from a nominal/string in a numeric value, for instance, using the StringToNominal/NominalToBinary filters, train a regression model using the corresponding dataset, evaluate an outcome as a numeric value and transfer this outcome back in the nominal value?
>
> The advantage: the outcome can be nominal or string and I don't have to specify it beforehand since everything is dynamic.
>
> Best, Alexander
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Alexander Osherenko
What is your opinion -- is it possible that regression outperforms supervised learning? If I formulate the problem as "regression relies on estimating relationships between variables that don't not necessarily represent all dimensions of the problem" vs "supervised learning fits itself around supervised data", I doubt it is possible.

Maybe, you know comparisons of performance of supervised classification and regression? I am aware of some references in "Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, Introduction to Information Retrieval, Cambridge University Press. 2008." https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf, p.347 but maybe there are more recent studies.
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Peter Reutemann
> What is your opinion -- is it possible that regression outperforms supervised
> learning? If I formulate the problem as "regression relies on estimating
> relationships between variables that don't not necessarily represent all
> dimensions of the problem" vs "supervised learning fits itself around
> supervised data", I doubt it is possible.
>
> Maybe, you know comparisons of performance of supervised classification and
> regression? I am aware of some references in "Christopher D. Manning,
> Prabhakar Raghavan and Hinrich Schütze, Introduction to Information
> Retrieval, Cambridge University Press. 2008."
> https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf, p.347 but
> maybe there are more recent studies.

A colleague of mine had better results using
ClassificationViaRegression for some projects. However, in my
projects, I haven't seen a major improvement compared to other
algorithms.

Cheers, Peter
--
Peter Reutemann
Dept. of Computer Science
University of Waikato, NZ
+64 (7) 858-5174
http://www.cms.waikato.ac.nz/~fracpete/
http://www.data-mining.co.nz/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Eibe Frank-2
Administrator
In reply to this post by Alexander Osherenko

> On 18/05/2017, at 8:48 PM, Alexander Osherenko <[hidden email]> wrote:
>
> What is your opinion -- is it possible that regression outperforms supervised
> learning? If I formulate the problem as "regression relies on estimating
> relationships between variables that don't not necessarily represent all
> dimensions of the problem" vs "supervised learning fits itself around
> supervised data", I doubt it is possible.
>
> Maybe, you know comparisons of performance of supervised classification and
> regression? I am aware of some references in "Christopher D. Manning,
> Prabhakar Raghavan and Hinrich Schütze, Introduction to Information
> Retrieval, Cambridge University Press. 2008."
> https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf, p.347 but
> maybe there are more recent studies.

Just a note on terminology: both regression and classification are considered supervised learning approaches.

Cheers,
Eibe
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Alexander Osherenko
Interesting. I thought, for supervised learning it is necessary to have a supervisor (somebody who sets the gold standard). Who is it in case of regression?

Best, Alexander

2017-05-18 23:02 GMT+01:00 Eibe Frank <[hidden email]>:

> On 18/05/2017, at 8:48 PM, Alexander Osherenko <[hidden email]> wrote:
>
> What is your opinion -- is it possible that regression outperforms supervised
> learning? If I formulate the problem as "regression relies on estimating
> relationships between variables that don't not necessarily represent all
> dimensions of the problem" vs "supervised learning fits itself around
> supervised data", I doubt it is possible.
>
> Maybe, you know comparisons of performance of supervised classification and
> regression? I am aware of some references in "Christopher D. Manning,
> Prabhakar Raghavan and Hinrich Schütze, Introduction to Information
> Retrieval, Cambridge University Press. 2008."
> https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf, p.347 but
> maybe there are more recent studies.

Just a note on terminology: both regression and classification are considered supervised learning approaches.

Cheers,
Eibe
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Regression - supervised learning

Eibe Frank-2
Administrator
The person who creates the numeric target values (i.e., the numeric “labels").

Cheers,
Eibe

> On 19/05/2017, at 5:40 PM, Alexander Osherenko <[hidden email]> wrote:
>
> Interesting. I thought, for supervised learning it is necessary to have a supervisor (somebody who sets the gold standard). Who is it in case of regression?
>
> Best, Alexander
>
> 2017-05-18 23:02 GMT+01:00 Eibe Frank <[hidden email]>:
>
> > On 18/05/2017, at 8:48 PM, Alexander Osherenko <[hidden email]> wrote:
> >
> > What is your opinion -- is it possible that regression outperforms supervised
> > learning? If I formulate the problem as "regression relies on estimating
> > relationships between variables that don't not necessarily represent all
> > dimensions of the problem" vs "supervised learning fits itself around
> > supervised data", I doubt it is possible.
> >
> > Maybe, you know comparisons of performance of supervised classification and
> > regression? I am aware of some references in "Christopher D. Manning,
> > Prabhakar Raghavan and Hinrich Schütze, Introduction to Information
> > Retrieval, Cambridge University Press. 2008."
> > https://nlp.stanford.edu/IR-book/pdf/irbookonlinereading.pdf, p.347 but
> > maybe there are more recent studies.
>
> Just a note on terminology: both regression and classification are considered supervised learning approaches.
>
> Cheers,
> Eibe
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html