Struggling with MLP outputs

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Struggling with MLP outputs

barmagan
Hello,
Some back ground first:
I code, but I am quite new to machine learning stuff. I have started with
Weka last week from (nearly) ground zero. I am trying to solve a real world
problem and I have figured it out how to use default Weka Logistic model
outputs(thanks to this list). Results are good but I want it better. So I
have recognized that MLP gives much more better results in Weka. But
unfortunately real world results are nonsense. I believe I am making a
mistake in implementing the outputs.

Problem definition:
I have 6 inputs (r,s,t,u,v,w), 2 output classes (True, False), 612
instances. Using default Weka MLP model.
There are 6 hidden nodes 4+2 (which comes default). Node2,3,4,5 are the
first stage, and Node0,1 are output stage.
Cross-validation result is %~93
But in application result of my code is always True.

As far as I have researched in internet each node should be calculated as:
SigmoidFunct(Sum(Wn*In)+Thresholdn)
So I have implement Weka  MLP outputs as following:
NODE2 = 1/(1+exp(-(WEIGHT2r*r +WEIGHT2s*s + ..... +WEIGHT2w*w +
threshold2)))
.........
.........
and finally I compare output nodes:
if (NODE1>NODE0) return TRUE;

Is this implementation correct? Or do I need more/better data?
I have checked the code. it looks ok. So I tried to summarize the code
above. If you say implementation is correct I will ask you to inspect my
data and code. If they are also ok all I can think that my data could be bad
and resulting thresholds saturate activation functions(sigmoid).

Thanks in advance,

B.Armagan







--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

barmagan
An additional note:
In a document I have read that threshold should subtracted from weighted
inputs. It says bias should be added which is bias=-threshold:
https://towardsdatascience.com/introducing-deep-learning-and-neural-networks-deep-learning-for-rookies-1-bd68f9cf5883

I also tried subtracting threshold but result didn't change. Output always
TRUE.



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

Eibe Frank-2
Administrator

Not entirely sure what you are trying to do. If you have the WEKA classifier object, and applied buildClassifier(Instances) to it, you can just use the classifyInstance(Instance) method or the distributionForInstance(Instance) method to obtain predictions.

 

Are you trying to reproduce the model from scratch based on the coefficients that WEKA’s MLP learning algorithm outputs?

 

Cheers,

Eibe

 

From: [hidden email]
Sent: Saturday, 9 February 2019 3:05 PM
To: [hidden email]
Subject: Re: [Wekalist] Struggling with MLP outputs

 

An additional note:

In a document I have read that threshold should subtracted from weighted

inputs. It says bias should be added which is bias=-threshold:

https://towardsdatascience.com/introducing-deep-learning-and-neural-networks-deep-learning-for-rookies-1-bd68f9cf5883

 

I also tried subtracting threshold but result didn't change. Output always

TRUE.

 

 

 

--

Sent from: http://weka.8497.n7.nabble.com/

_______________________________________________

Wekalist mailing list

Send posts to: [hidden email]

To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist

List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

 


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

barmagan
Hi Eibe,
Thank you for response. I have  this data
<https://drive.google.com/file/d/1GHsiCfi5dsGF79LUw7hWlzO2VTblVEwM/view>  .
In weka explorer gui I have run MultilayerPerceptron classifier with default
settings and it gave me weight and threshold values( here is the output
<https://drive.google.com/file/d/1GlFDbQjGB3KX6vbvho9F8q1df9ZZMM_i/view>  ).

If I didn't get it wrong, these are parameters for MLP classifier model. And
if I give new instances to MLP model it should classify it as FALSE/TRUE(in
my case). So I have written a classifier model with these parameters(weights
and thresholds).And  here is the code <http://tpcg.io/4QEeXv>  . When I give
new/old instances I always get TRUE. Weka's MLP classifier model might be
different than my code.

I used weka only for generating the weights and thresholds. I thought it
would be easier to write my own code, so I didn't use weka. I tried look at
the weka source code in order to find the difference, but I got lost. If it
is possible can you please check my code and tell the difference?



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

barmagan
In reply to this post by Eibe Frank-2
And yes. I think I am trying to reproduce the model.
The following link is a simplified version of my code producing same
problematic results. (More readable than the previous link)

Can you please tell me the problem in my code?



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

Eibe Frank-2
Administrator
MultilayerPerceptron in WEKA rescales all the inputs to the [-1, 1] interval. Are you doing that in your code?

Cheers,
Eibe

> On 12/02/2019, at 1:19 AM, barmagan <[hidden email]> wrote:
>
> And yes. I think I am trying to reproduce the model.
> The following link is a simplified version of my code producing same
> problematic results. (More readable than the previous link)
>
> Can you please tell me the problem in my code?
>
>
>
> --
> Sent from: http://weka.8497.n7.nabble.com/
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

barmagan
Hi Eibe,
No I didn't scale inputs. I will try that. But,
Should I only scale instance inputs? Or should I also scale node outputs
(which are also inputs for last layer)? My current activation function is
sigmoid [0,-1].
A- Should I leave it as sigmoid?
B- or Should I scale (weight*input-threshold) to [-1,1] range?
C- Should I scale sigmoid output [0,1] to [-1,1]?

Note: I use real-time updated instance variables for input (on an Android
phone). As an embedded C developer, android environment is already very
complicated for me. Weka manual says, data is loaded from files or database.
So in order to keep things simple I preferred rewriting the classifier
myself instead of using weka directly.
Do you think it would be easier to use weka or re-write the classifier?

Thanks

B.Armagan



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

Eibe Frank-3
Perhaps it's easier to hard-code the classifier based on your current approach. It is a shame that the MultilayerPerceptron is not Sourcable. Some other WEKA classifiers are.

AFAIK, only the input features (i.e., attributes) are scaled to [-1, 1], i.e., only the data that comes into the network.

Cheers,
Eibe

On Tue, Feb 12, 2019 at 10:00 PM barmagan <[hidden email]> wrote:
Hi Eibe,
No I didn't scale inputs. I will try that. But,
Should I only scale instance inputs? Or should I also scale node outputs
(which are also inputs for last layer)? My current activation function is
sigmoid [0,-1].
A- Should I leave it as sigmoid?
B- or Should I scale (weight*input-threshold) to [-1,1] range?
C- Should I scale sigmoid output [0,1] to [-1,1]?

Note: I use real-time updated instance variables for input (on an Android
phone). As an embedded C developer, android environment is already very
complicated for me. Weka manual says, data is loaded from files or database.
So in order to keep things simple I preferred rewriting the classifier
myself instead of using weka directly.
Do you think it would be easier to use weka or re-write the classifier?

Thanks

B.Armagan



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Struggling with MLP outputs

barmagan
Hi Eibe,
Thank you very much for your help.
I have scaled inputs and It is working perfectly fine now :-)

Regards,

B.Armagan



--
Sent from: http://weka.8497.n7.nabble.com/
_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html