Convolutional Neural Network using D4LjMlpClassifier

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Convolutional Neural Network using D4LjMlpClassifier

George J
Apologies if you are seeing this again.

Trying to get CNN running using D4LjMLPClassifier for image classification. Can't initialise it at all. Just starts and stops immediately.
There is no output error from WEKA and from the D4LjMLPClassifier no log file is created. Thought about the debug info but there is no output there either. Went to weka's Explorer with no luck.

Any ideas?

CLI command:

java -Xmx8G -cp "C:\Program Files\Weka-3-8\weka.jar" weka.Run .Dl4jMlpClassifier \
-S 1 \
-iterator "weka.dl4j.iterators.ImageDataSetIterator -height 28 -imagesLocation  filepath_to_images -numChannels 1 -bs 1 -width 28" \
-layer "weka.dl4j.layers.OutputLayer -activation \"weka.dl4j.activations.ActivationSoftmax \" -adamMeanDecay 0.9 -adamVarDecay 0.999 -biasInit 1.0 -l1Bias 0.0 -l2Bias 0.0 -blr 0.01 \
-dist \"weka.dl4j.distribution.NormalDistribution -mean 0.001 -std 1.0\" -dropout 0.0 -epsilon 1.0E-6 -gradientNormalization None -gradNormThreshold 1.0 -L1 0.0 -L2 0.0
-name \"Output layer\" -lr 0.01 -lossFn \"weka.dl4j.lossfunctions.LossMCXENT \" -momentum 0.9 -rho 0.0 -rmsDecay 0.95 -updater NESTEROVS -weightInit XAVIER" \
-logFile C:\Users\Modelling\Desktop -numEpochs 10 -algorithm STOCHASTIC_GRADIENT_DESCENT \
-output-debug-info
Reply | Threaded
Open this post in threaded view
|

Re: Convolutional Neural Network using D4LjMlpClassifier

Eibe Frank-2
Administrator
Did you manage to run the mini MNIST example in the $WEKA_HOME/wekafiles/wekaDeeplearning4jCore/data folder?

Which packages did you install? The three packages for the CPU version?

What’s your OS and which version of Java are you using?

Your command-line is basically fine, but you haven’t provided an actual training file using -t.

Also, it turns out that you need to have a ConvolutionLayer, SubsamplingLayer, or BatchNormalization layer as the first layer when you use the ImageDataSetIterator.

Here is a simplified version of your command-line that works for me:

java weka.Run .Dl4jMlpClassifier -t mnist-minimal.arff -iterator "weka.dl4j.iterators.ImageDataSetIterator -height 28 -imagesLocation mnist-minimal -numChannels 1 -bs 1 -width 28" -layer "weka.dl4j.layers.ConvolutionLayer -nFilters 5" -layer "weka.dl4j.layers.OutputLayer" -numEpochs 10

Cheers,
Eibe

> On 29/05/2017, at 6:15 AM, cibic89 <[hidden email]> wrote:
>
> Apologies if you are seeing this again.
>
> Trying to get CNN running using D4LjMLPClassifier for image classification.
> Can't initialise it at all. Just starts and stops immediately.
> There is no output error from WEKA and from the D4LjMLPClassifier no log
> file is created. Thought about the debug info but there is no output there
> either. Went to weka's Explorer with no luck.
>
> Any ideas?
>
> CLI command:
>
> java -Xmx8G -cp "C:\Program Files\Weka-3-8\weka.jar" weka.Run
> .Dl4jMlpClassifier \
> -S 1 \
> -iterator "weka.dl4j.iterators.ImageDataSetIterator -height 28
> -imagesLocation  filepath_to_images -numChannels 1 -bs 1 -width 28" \
> -layer "weka.dl4j.layers.OutputLayer -activation
> \"weka.dl4j.activations.ActivationSoftmax \" -adamMeanDecay 0.9
> -adamVarDecay 0.999 -biasInit 1.0 -l1Bias 0.0 -l2Bias 0.0 -blr 0.01 \
> -dist \"weka.dl4j.distribution.NormalDistribution -mean 0.001 -std 1.0\"
> -dropout 0.0 -epsilon 1.0E-6 -gradientNormalization None -gradNormThreshold
> 1.0 -L1 0.0 -L2 0.0
> -name \"Output layer\" -lr 0.01 -lossFn \"weka.dl4j.lossfunctions.LossMCXENT
> \" -momentum 0.9 -rho 0.0 -rmsDecay 0.95 -updater NESTEROVS -weightInit
> XAVIER" \
> -logFile C:\Users\Modelling\Desktop -numEpochs 10 -algorithm
> STOCHASTIC_GRADIENT_DESCENT \
> -output-debug-info
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Convolutional-Neural-Network-using-D4LjMlpClassifier-tp40774.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Convolutional Neural Network using D4LjMlpClassifier

George J
This post was updated on .
I did try the mnist example with the same issue.

Went for the CPU version and installed these three packages:
wekaDeeplearning4jCore 1.0.6
wekaDeeplearning4jCPU 1.0.3
wekaDeeplearning4jCPULibs 1.0.1

OS and java: Windows 10 and JDK 8 (v1.8). Fresh install.

Sorry, I removed the training parameter after I pasted the command above.


Tried the command you gave in terminal and got this:

[WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and wekaDeeplearning4jCPULibs

Weka exception: Illegal options: -numChannels 1 -bs 1 -width 28 -layer weka.dl4j.layers.ConvolutionLayer -nFilters 5 -layer weka.dl4j.layers.OutputLayer -numEpochs 10

These options should be valid but they don't appear in the classifier-specific help output?!?!


In Explorer, the same issue as previously. Starts and stops immediately but this time got an output:

[WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and wekaDeeplearning4jCPULibs
Problem with 'org.deeplearning4j.nn.conf.distribution.Distribution': java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.distribution.Distribut
Problem with 'org.deeplearning4j.nn.conf.layers.Layer': java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.layers.Layer
Problem with 'org.nd4j.linalg.lossfunctions.ILossFunction': java.lang.ClassNotFoundException: org.nd4j.linalg.lossfunctions.ILossFunction
java.lang.NullPointerException
        weka.core.Instances.<init>(Instances.java:179)
...
May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation updaterValidation
WARNING: Layer "Output layer" adamVarDecay is set but will not be applied unless the updater is set to Adam.
...
May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation updaterValidation
WARNING: Layer "Output layer" rho is set but will not be applied unless the updater is set to ADADELTA.
...
May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation updaterValidation
WARNING: Layer "Output layer" rmsdecay is set but will not be applied unless the updater is set to RMSPROP.
...

Cheers,
George
Reply | Threaded
Open this post in threaded view
|

Re: Convolutional Neural Network using D4LjMlpClassifier

Eibe Frank-2
Administrator
Sorry, it looks like those package versions are broken. I'll look into this some more tomorrow.

Cheers,
Eibe

> On 30 May 2017, at 18:47, cibic89 <[hidden email]> wrote:
>
> I did try the mnist example with the same issue.
>
> Went for the CPU version and installed these three packages:
> wekaDeeplearning4jCore 1.0.6
> wekaDeeplearning4jCPU 1.0.3
> wekaDeeplearning4jCPULibs 1.0.1
>
> OS and java: Windows 10 and JDK 8 (v1.8). Fresh install.
>
> Sorry, I changed the file path before sending but I did have the right one
> set each time I attempted to run D4LjMlpClassifier.
>
>
> Tried the command you gave in terminal and got this:
>
> [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
> wekaDeeplearning4jCPULibs
>
> Weka exception: Illegal options: -numChannels 1 -bs 1 -width 28 -layer
> weka.dl4j.layers.ConvolutionLayer -nFilters 5 -layer
> weka.dl4j.layers.OutputLayer -numEpochs 10
>
> These options should be valid but they don't appear in the
> classifier-specific help output?!?!
>
>
> In Explorer, the same issue as previously. Starts and stops immediately but
> this time got an output:
>
> [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
> wekaDeeplearning4jCPULibs
> Problem with 'org.deeplearning4j.nn.conf.distribution.Distribution':
> java.lang.ClassNotFoundException:
> org.deeplearning4j.nn.conf.distribution.Distribut
> Problem with 'org.deeplearning4j.nn.conf.layers.Layer':
> java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.layers.Layer
> Problem with 'org.nd4j.linalg.lossfunctions.ILossFunction':
> java.lang.ClassNotFoundException:
> org.nd4j.linalg.lossfunctions.ILossFunction
> java.lang.NullPointerException
>        weka.core.Instances.<init>(Instances.java:179)
> ...
> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
> updaterValidation
> WARNING: Layer "Output layer" adamVarDecay is set but will not be applied
> unless the updater is set to Adam.
> ...
> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
> updaterValidation
> WARNING: Layer "Output layer" rho is set but will not be applied unless the
> updater is set to ADADELTA.
> ...
> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
> updaterValidation
> WARNING: Layer "Output layer" rmsdecay is set but will not be applied unless
> the updater is set to RMSPROP.
> ...
>
> Cheers,
> George
>
>
>
> --
> View this message in context: http://weka.8497.n7.nabble.com/Convolutional-Neural-Network-using-D4LjMlpClassifier-tp40774p40799.html
> Sent from the WEKA mailing list archive at Nabble.com.
> _______________________________________________
> Wekalist mailing list
> Send posts to: [hidden email]
> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Convolutional Neural Network using D4LjMlpClassifier

Mark Hall
I've uploaded a fixed version of wekaDeeplearning4jCore1.0.6 to sourceforge. Please re-install this package and let us know if it works properly for you now.

Cheers,
Mark.

On 30/05/17, 12:40 PM, "Eibe Frank" <[hidden email] on behalf of [hidden email]> wrote:

    Sorry, it looks like those package versions are broken. I'll look into this some more tomorrow.
   
    Cheers,
    Eibe
   
    > On 30 May 2017, at 18:47, cibic89 <[hidden email]> wrote:
    >
    > I did try the mnist example with the same issue.
    >
    > Went for the CPU version and installed these three packages:
    > wekaDeeplearning4jCore 1.0.6
    > wekaDeeplearning4jCPU 1.0.3
    > wekaDeeplearning4jCPULibs 1.0.1
    >
    > OS and java: Windows 10 and JDK 8 (v1.8). Fresh install.
    >
    > Sorry, I changed the file path before sending but I did have the right one
    > set each time I attempted to run D4LjMlpClassifier.
    >
    >
    > Tried the command you gave in terminal and got this:
    >
    > [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
    > wekaDeeplearning4jCPULibs
    >
    > Weka exception: Illegal options: -numChannels 1 -bs 1 -width 28 -layer
    > weka.dl4j.layers.ConvolutionLayer -nFilters 5 -layer
    > weka.dl4j.layers.OutputLayer -numEpochs 10
    >
    > These options should be valid but they don't appear in the
    > classifier-specific help output?!?!
    >
    >
    > In Explorer, the same issue as previously. Starts and stops immediately but
    > this time got an output:
    >
    > [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
    > wekaDeeplearning4jCPULibs
    > Problem with 'org.deeplearning4j.nn.conf.distribution.Distribution':
    > java.lang.ClassNotFoundException:
    > org.deeplearning4j.nn.conf.distribution.Distribut
    > Problem with 'org.deeplearning4j.nn.conf.layers.Layer':
    > java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.layers.Layer
    > Problem with 'org.nd4j.linalg.lossfunctions.ILossFunction':
    > java.lang.ClassNotFoundException:
    > org.nd4j.linalg.lossfunctions.ILossFunction
    > java.lang.NullPointerException
    >        weka.core.Instances.<init>(Instances.java:179)
    > ...
    > May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
    > updaterValidation
    > WARNING: Layer "Output layer" adamVarDecay is set but will not be applied
    > unless the updater is set to Adam.
    > ...
    > May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
    > updaterValidation
    > WARNING: Layer "Output layer" rho is set but will not be applied unless the
    > updater is set to ADADELTA.
    > ...
    > May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
    > updaterValidation
    > WARNING: Layer "Output layer" rmsdecay is set but will not be applied unless
    > the updater is set to RMSPROP.
    > ...
    >
    > Cheers,
    > George
    >
    >
    >
    > --
    > View this message in context: http://weka.8497.n7.nabble.com/Convolutional-Neural-Network-using-D4LjMlpClassifier-tp40774p40799.html
    > Sent from the WEKA mailing list archive at Nabble.com.
    > _______________________________________________
    > Wekalist mailing list
    > Send posts to: [hidden email]
    > List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
    > List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
   
    _______________________________________________
    Wekalist mailing list
    Send posts to: [hidden email]
    List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
    List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
   


_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: Convolutional Neural Network using D4LjMlpClassifier

Eibe Frank-2
Administrator
It appears to work now. I have installed and tested wekaDeeplearning4jCore 1.0.6 for WEKA 3.8.1 on my Mac and on Windows 10.

There are a couple of issues left:

1) When the packages are installed, it looks like the installation failed and the packages cannot be loaded. The following output appears in the console:

...
[DefaultPackageManager] Tmp file: /var/folders/l1/5vsyc_316ps69_s_s4t8lbnw0000gp/T/wekaDeeplearning4jCPU1.0.3333403925918107335.zip
[DefaultPackageManager] Installing: Description.props
[Weka] Integrity: removing classloader for: wekaDeeplearning4jCPU
Problem with 'org.deeplearning4j.nn.conf.distribution.Distribution': java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.distribution.Distribution
Problem with 'org.deeplearning4j.nn.conf.layers.Layer': java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.layers.Layer
Problem with 'weka.dl4j.iterators.AbstractDataSetIterator': java.lang.ClassNotFoundException: weka.dl4j.iterators.AbstractDataSetIterator
Problem with 'org.nd4j.linalg.lossfunctions.ILossFunction': java.lang.ClassNotFoundException: org.nd4j.linalg.lossfunctions.ILossFunction

However, when WEKA is restarted after installation, everything appears to work as expected.

2) I forgot to update the README containing the example convolutional network configuration in the data folder of the package so that it works with wekaDeeplearning4jCore 1.0.6/1.1.6. A simplified configuration that works is

weka.classifiers.functions.Dl4jMlpClassifier -S 1 -iterator "weka.dl4j.iterators.ImageDataSetIterator -imagesLocation wekaDeeplearning4j/data/mnist-minimal -bs 1 -numChannels 1  -width 28 -height 28 " -layer "weka.dl4j.layers.ConvolutionLayer -activation \"weka.dl4j.activations.ActivationIdentity\" -lr 0.0001 -blr 0.0001 -nFilters 5 -kernelSizeX 5 -kernelSizeY 5 -paddingX 0 -paddingY 0 -strideX 1 -strideY 1" -layer "weka.dl4j.layers.DenseLayer -nOut 20 -activation \"weka.dl4j.activations.ActivationReLU\" -lr 0.0001 -blr 0.0001" -layer "weka.dl4j.layers.OutputLayer -activation \"weka.dl4j.activations.ActivationSoftmax\" -lossFn weka.dl4j.lossfunctions.LossMCXENT -lr 0.0001 -blr 0.0001" -numEpochs 100

I have committed the new version of the README into the SVN repository now:

https://svn.cms.waikato.ac.nz/svn/weka/trunk/packages/internal/wekaDeeplearning4jCore/data/README

George, regarding valid command-line options for the layer specification, etc., you currently have to look at the option annotations in the source code, e.g.,

https://svn.cms.waikato.ac.nz/svn/weka/branches/stable-3-8/packages/internal/wekaDeeplearning4jCore/src/main/java/weka/dl4j/layers/DenseLayer.java

Cheers,
Eibe

> On 31/05/2017, at 2:51 AM, Mark Hall <[hidden email]> wrote:
>
> I've uploaded a fixed version of wekaDeeplearning4jCore1.0.6 to sourceforge. Please re-install this package and let us know if it works properly for you now.
>
> Cheers,
> Mark.
>
> On 30/05/17, 12:40 PM, "Eibe Frank" <[hidden email] on behalf of [hidden email]> wrote:
>
>    Sorry, it looks like those package versions are broken. I'll look into this some more tomorrow.
>
>    Cheers,
>    Eibe
>
>> On 30 May 2017, at 18:47, cibic89 <[hidden email]> wrote:
>>
>> I did try the mnist example with the same issue.
>>
>> Went for the CPU version and installed these three packages:
>> wekaDeeplearning4jCore 1.0.6
>> wekaDeeplearning4jCPU 1.0.3
>> wekaDeeplearning4jCPULibs 1.0.1
>>
>> OS and java: Windows 10 and JDK 8 (v1.8). Fresh install.
>>
>> Sorry, I changed the file path before sending but I did have the right one
>> set each time I attempted to run D4LjMlpClassifier.
>>
>>
>> Tried the command you gave in terminal and got this:
>>
>> [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
>> wekaDeeplearning4jCPULibs
>>
>> Weka exception: Illegal options: -numChannels 1 -bs 1 -width 28 -layer
>> weka.dl4j.layers.ConvolutionLayer -nFilters 5 -layer
>> weka.dl4j.layers.OutputLayer -numEpochs 10
>>
>> These options should be valid but they don't appear in the
>> classifier-specific help output?!?!
>>
>>
>> In Explorer, the same issue as previously. Starts and stops immediately but
>> this time got an output:
>>
>> [WekaPackageManager] Added a dependency between wekaDeeplearning4jCore and
>> wekaDeeplearning4jCPULibs
>> Problem with 'org.deeplearning4j.nn.conf.distribution.Distribution':
>> java.lang.ClassNotFoundException:
>> org.deeplearning4j.nn.conf.distribution.Distribut
>> Problem with 'org.deeplearning4j.nn.conf.layers.Layer':
>> java.lang.ClassNotFoundException: org.deeplearning4j.nn.conf.layers.Layer
>> Problem with 'org.nd4j.linalg.lossfunctions.ILossFunction':
>> java.lang.ClassNotFoundException:
>> org.nd4j.linalg.lossfunctions.ILossFunction
>> java.lang.NullPointerException
>>       weka.core.Instances.<init>(Instances.java:179)
>> ...
>> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
>> updaterValidation
>> WARNING: Layer "Output layer" adamVarDecay is set but will not be applied
>> unless the updater is set to Adam.
>> ...
>> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
>> updaterValidation
>> WARNING: Layer "Output layer" rho is set but will not be applied unless the
>> updater is set to ADADELTA.
>> ...
>> May 29, 2017 5:21:52 PM org.deeplearning4j.nn.conf.layers.LayerValidation
>> updaterValidation
>> WARNING: Layer "Output layer" rmsdecay is set but will not be applied unless
>> the updater is set to RMSPROP.
>> ...
>>
>> Cheers,
>> George
>>
>>
>>
>> --
>> View this message in context: http://weka.8497.n7.nabble.com/Convolutional-Neural-Network-using-D4LjMlpClassifier-tp40774p40799.html
>> Sent from the WEKA mailing list archive at Nabble.com.
>> _______________________________________________
>> Wekalist mailing list
>> Send posts to: [hidden email]
>> List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
>> List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
>    _______________________________________________
>    Wekalist mailing list
>    Send posts to: [hidden email]
>    List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
>    List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
>
>
>

_______________________________________________
Wekalist mailing list
Send posts to: [hidden email]
List info and subscription status: https://list.waikato.ac.nz/mailman/listinfo/wekalist
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html