DeepLearning Exception on RemoteEngine

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

DeepLearning Exception on RemoteEngine

Joel Ratsaby-2
I added using the PackageManager DeepLearning. 

When I run DeepLearning4j from the Experimenter in standalone there is no problem.

But if I run it on a RemoteEngine I get the Exception listed below (in yellow) although I  added the following to the classpath that calls the Experimenter on the remote machine:

/XYZ/wekafiles/packages/wekaDeeplearning4j/*:/XYZ/wekafiles/packages/wekaDeeplearning4j/lib/*:

I can see that the class org.nd4j.linalg.factory.Nd4j    is located in the jar file
nd4j-api-1.0.0-beta5.jar which is under the  /wekafiles/packages/wekaDeeplearning4j/lib/ folder.

Any advise?




[INFO ] 20:14:23.330 [Thread-24] weka.classifiers.functions.Dl4jMlpClassifier - Building on 90 training instances
Task id 1588007663170:2026456888 Unknown initialization error
java.lang.NoClassDefFoundError: Could not initialize class org.nd4j.linalg.factory.Nd4j
weka.classifiers.functions.dl4j.Utils.instancesToDataSet(Utils.java:64)
weka.dl4j.iterators.instance.DefaultInstanceIterator.getDataSetIterator(DefaultInstanceIterator.java:66)
weka.dl4j.iterators.instance.AbstractInstanceIterator.getDataSetIterator(AbstractInstanceIterator.java:59)
weka.classifiers.functions.Dl4jMlpClassifier.getDataSetIterator(Dl4jMlpClassifier.java:1069)
weka.classifiers.functions.Dl4jMlpClassifier.getDataSetIterator(Dl4jMlpClassifier.java:1121)
weka.classifiers.functions.Dl4jMlpClassifier.getFirstBatchFeatures(Dl4jMlpClassifier.java:1449)
weka.classifiers.functions.Dl4jMlpClassifier.createModel(Dl4jMlpClassifier.java:1294)
weka.classifiers.functions.Dl4jMlpClassifier.finishClassifierInitialization(Dl4jMlpClassifier.java:957)
weka.classifiers.functions.Dl4jMlpClassifier.initializeClassifier(Dl4jMlpClassifier.java:899)
weka.classifiers.functions.Dl4jMlpClassifier.buildClassifier(Dl4jMlpClassifier.java:816)
weka.classifiers.meta.FilteredClassifier.buildClassifier(FilteredClassifier.java:811)
weka.experiment.ClassifierSplitEvaluator.getResult(ClassifierSplitEvaluator.java:836)
weka.experiment.CrossValidationResultProducer.doRun(CrossValidationResultProducer.java:393)
weka.experiment.Experiment.nextIteration(Experiment.java:575)
weka.experiment.RemoteExperimentSubTask.execute(RemoteExperimentSubTask.java:106)
weka.experiment.RemoteEngine$2.run(RemoteEngine.java:201)

at weka.classifiers.functions.dl4j.Utils.instancesToDataSet(Utils.java:64)
at weka.dl4j.iterators.instance.DefaultInstanceIterator.getDataSetIterator(DefaultInstanceIterator.java:66)
at weka.dl4j.iterators.instance.AbstractInstanceIterator.getDataSetIterator(AbstractInstanceIterator.java:59)
at weka.classifiers.functions.Dl4jMlpClassifier.getDataSetIterator(Dl4jMlpClassifier.java:1069)
at weka.classifiers.functions.Dl4jMlpClassifier.getDataSetIterator(Dl4jMlpClassifier.java:1121)
at weka.classifiers.functions.Dl4jMlpClassifier.getFirstBatchFeatures(Dl4jMlpClassifier.java:1449)
at weka.classifiers.functions.Dl4jMlpClassifier.createModel(Dl4jMlpClassifier.java:1294)
at weka.classifiers.functions.Dl4jMlpClassifier.finishClassifierInitialization(Dl4jMlpClassifier.java:957)
at weka.classifiers.functions.Dl4jMlpClassifier.initializeClassifier(Dl4jMlpClassifier.java:899)
at weka.classifiers.functions.Dl4jMlpClassifier.buildClassifier(Dl4jMlpClassifier.java:816)
at weka.classifiers.meta.FilteredClassifier.buildClassifier(FilteredClassifier.java:811)
at weka.experiment.ClassifierSplitEvaluator.getResult(ClassifierSplitEvaluator.java:836)
at weka.experiment.CrossValidationResultProducer.doRun(CrossValidationResultProducer.java:393)
at weka.experiment.Experiment.nextIteration(Experiment.java:575)
at weka.experiment.RemoteExperimentSubTask.execute(RemoteExperimentSubTask.java:106)
at weka.experiment.RemoteEngine$2.run(RemoteEngine.java:201)
Detailed message Could not initialize class org.nd4j.linalg.factory.Nd4j
Exception in thread "Thread-24" java.lang.NullPointerException
weka.experiment.RemoteEngine$2.run(RemoteEngine.java:223)

at weka.experiment.RemoteEngine$2.run(RemoteEngine.java:223)
Finished/failed Task id : 1588007663170:2026456888 checked by client. Removing.






_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Michael Hall


On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

My usual stuff doesn’t seem to find it either. Maybe it is lazy loaded.
In the package lib directory I do see 15 nd4j jars. 3 appear to be platform specific. 
Would you need to add some or all of those to a class path? 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Michael Hall


On Apr 27, 2020, at 7:43 PM, Michael Hall <[hidden email]> wrote:



On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

My usual stuff doesn’t seem to find it either. Maybe it is lazy loaded.
In the package lib directory I do see 15 nd4j jars. 3 appear to be platform specific. 
Would you need to add some or all of those to a class path? 

Sorry I didn’t look closely enough at yours to see you had already found it’s jar. I would still wonder the same thing though. Maybe loading this class requires loading a dependent class in another jar file? It throws the error on the original class even though it’s something else it can’t find maybe?

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Michael Hall
In reply to this post by Joel Ratsaby-2


On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

Based on this…

java -cp halfpipe.jar:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/lib/*:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/*:/Applications/weka-3-9-3/weka.jar org.cmdline.cmds.showc org.nd4j.linalg.factory.Nd4j
showc: invoked with: org.nd4j.linalg.factory.Nd4j
java.lang.ExceptionInInitializerError
at org.apache.logging.log4j.util.StackLocatorUtil.<clinit>(StackLocatorUtil.java:28)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:42)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.nd4j.linalg.factory.Nd4jBackend.<clinit>(Nd4jBackend.java:64)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5152)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:182)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:340)
at org.cmdline.cmds.showc.main(showc.java:78)
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getStackWalkerWithClassReference")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
at java.base/java.lang.StackWalker.checkPermission(StackWalker.java:424)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:365)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:336)
at org.apache.logging.log4j.util.StackLocator.<clinit>(StackLocator.java:28)
... 12 more

Since it seems to be loading correctly I think there is something not quite right about the class path. Whatever the XYZ is? 


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Joel Ratsaby-2
 XYZ is just  like your  /Users/mjh/.

Here is my classpath on the remote engine machine:

java  -Xmx256m -XX:+UseSerialGC \
 -cp "/home/ratsaby/MY_HSQLDB/hsqldb.jar:/home/ratsaby/WEKA/jars/lib/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/lib/*:/home/ratsaby/WEKA/jars/*" \
 -Djava.security.policy=/home/ratsaby/WEKA/remote_engine/remote.policy \

I created the  WEKA jar from the sources (I downloaded weka 3.8.4). (Note again that the same classpath works well for standalone, namely, for running DL4j on Experimenter without RemoteEngine. the DL4j problem just arises when I use RemoteEngine).

Should I try it with weka-3-9?

I don't understand your conclusion. Did you find the reason? If so what do I need to add to the classpath of the Remoteengine? 




On Tue, 28 Apr 2020 at 04:33, Michael Hall <[hidden email]> wrote:


On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

Based on this…

java -cp halfpipe.jar:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/lib/*:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/*:/Applications/weka-3-9-3/weka.jar org.cmdline.cmds.showc org.nd4j.linalg.factory.Nd4j
showc: invoked with: org.nd4j.linalg.factory.Nd4j
java.lang.ExceptionInInitializerError
at org.apache.logging.log4j.util.StackLocatorUtil.<clinit>(StackLocatorUtil.java:28)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:42)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.nd4j.linalg.factory.Nd4jBackend.<clinit>(Nd4jBackend.java:64)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5152)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:182)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:340)
at org.cmdline.cmds.showc.main(showc.java:78)
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getStackWalkerWithClassReference")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
at java.base/java.lang.StackWalker.checkPermission(StackWalker.java:424)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:365)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:336)
at org.apache.logging.log4j.util.StackLocator.<clinit>(StackLocator.java:28)
... 12 more

Since it seems to be loading correctly I think there is something not quite right about the class path. Whatever the XYZ is? 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


--
Prof. Joel Ratsaby
Electrical and Electronics Engineering Dept.
Ariel University
ARIEL 40700
ISRAEL



_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Eibe Frank-2
Administrator
Hi Joel,

I got the same error as you, had a look at the error messages, and realised that the remote_engine.policy file required additional permissions. One by one, re-running the remote engine many times, I managed to eliminate all errors that explicitly mention permissions by adding the following permissions:

  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/nd4j-native-1.0.0-beta5-linux-x86_64.jar/org/nd4j/nativeblas/linux-x86_64/libjnind4jcpu.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libgomp.so.1";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libiomp5.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libmklml_intel.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libmkldnn.so.0";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/nd4j-native-1.0.0-beta5-linux-x86_64.jar/org/nd4j/nativeblas/linux-x86_64/libnd4jcpu.so";
  permission java.lang.RuntimePermission
             "getStackWalkerWithClassReference";
  permission java.lang.RuntimePermission
             "createSecurityManager";
  permission "java.io.FilePermission"
             "/usr/lib/-", "read";
  permission "java.io.FilePermission"
             "/usr/lib64/-", "read";
  permission "java.io.FilePermission"
             "/lib64/-", "read";
  permission "java.io.FilePermission"
             "/lib/-", "read";
  permission "java.io.FilePermission"
             "/home/eibe/wekafiles/-", "write";
  permission "java.io.FilePermission"
             "/home/eibe/.javacpp/-", "read,write,execute,delete,readlink";
  permission "javax.management.MBeanServerPermission"
             "createMBeanServer";
  permission "javax.management.MBeanPermission"
             "-#-[-]", "queryNames";

Unfortunately, the original error remained and the error messages produced by the remote engine still contained the line

Task id 1588075306911:1378015012 Failed! Check remote policy file

So it looked like other permissions needed to be set but the corresponding exceptions were swallowed up somewhere in the code. As a last resort, which WORKED, I put the following into the remote_engine.policy file (and nothing else):

grant {
    permission java.security.AllPermission "", "";
    };

Make sure only trusted machines can connect to the machine with the remote engine at the port used by the remote engine. (This is advisable anyway.)

Cheers,
Eibe

On 28/04/2020, at 10:30 PM, Joel Ratsaby <[hidden email]> wrote:

 XYZ is just  like your  /Users/mjh/.

Here is my classpath on the remote engine machine:

java  -Xmx256m -XX:+UseSerialGC \
 -cp "/home/ratsaby/MY_HSQLDB/hsqldb.jar:/home/ratsaby/WEKA/jars/lib/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/lib/*:/home/ratsaby/WEKA/jars/*" \
 -Djava.security.policy=/home/ratsaby/WEKA/remote_engine/remote.policy \

I created the  WEKA jar from the sources (I downloaded weka 3.8.4). (Note again that the same classpath works well for standalone, namely, for running DL4j on Experimenter without RemoteEngine. the DL4j problem just arises when I use RemoteEngine).

Should I try it with weka-3-9?

I don't understand your conclusion. Did you find the reason? If so what do I need to add to the classpath of the Remoteengine? 




On Tue, 28 Apr 2020 at 04:33, Michael Hall <[hidden email]> wrote:


On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

Based on this…

java -cp halfpipe.jar:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/lib/*:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/*:/Applications/weka-3-9-3/weka.jar org.cmdline.cmds.showc org.nd4j.linalg.factory.Nd4j
showc: invoked with: org.nd4j.linalg.factory.Nd4j
java.lang.ExceptionInInitializerError
at org.apache.logging.log4j.util.StackLocatorUtil.<clinit>(StackLocatorUtil.java:28)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:42)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.nd4j.linalg.factory.Nd4jBackend.<clinit>(Nd4jBackend.java:64)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5152)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:182)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:340)
at org.cmdline.cmds.showc.main(showc.java:78)
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getStackWalkerWithClassReference")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
at java.base/java.lang.StackWalker.checkPermission(StackWalker.java:424)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:365)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:336)
at org.apache.logging.log4j.util.StackLocator.<clinit>(StackLocator.java:28)
... 12 more

Since it seems to be loading correctly I think there is something not quite right about the class path. Whatever the XYZ is? 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


--
Prof. Joel Ratsaby
Electrical and Electronics Engineering Dept.
Ariel University
ARIEL 40700
ISRAEL


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html
Reply | Threaded
Open this post in threaded view
|

Re: DeepLearning Exception on RemoteEngine

Joel Ratsaby-2
Eibe
Thanks very much, I added the java.security.AllPermission and it works.
Cheers,
Joel

On Tue, 28 Apr 2020 at 15:17, Eibe Frank <[hidden email]> wrote:
Hi Joel,

I got the same error as you, had a look at the error messages, and realised that the remote_engine.policy file required additional permissions. One by one, re-running the remote engine many times, I managed to eliminate all errors that explicitly mention permissions by adding the following permissions:

  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/nd4j-native-1.0.0-beta5-linux-x86_64.jar/org/nd4j/nativeblas/linux-x86_64/libjnind4jcpu.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libgomp.so.1";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libiomp5.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libmklml_intel.so";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/mkl-dnn-0.20-1.5.1-linux-x86_64.jar/org/bytedeco/mkldnn/linux-x86_64/libmkldnn.so.0";
  permission "java.lang.RuntimePermission"
             "loadLibrary./home/eibe/.javacpp/cache/nd4j-native-1.0.0-beta5-linux-x86_64.jar/org/nd4j/nativeblas/linux-x86_64/libnd4jcpu.so";
  permission java.lang.RuntimePermission
             "getStackWalkerWithClassReference";
  permission java.lang.RuntimePermission
             "createSecurityManager";
  permission "java.io.FilePermission"
             "/usr/lib/-", "read";
  permission "java.io.FilePermission"
             "/usr/lib64/-", "read";
  permission "java.io.FilePermission"
             "/lib64/-", "read";
  permission "java.io.FilePermission"
             "/lib/-", "read";
  permission "java.io.FilePermission"
             "/home/eibe/wekafiles/-", "write";
  permission "java.io.FilePermission"
             "/home/eibe/.javacpp/-", "read,write,execute,delete,readlink";
  permission "javax.management.MBeanServerPermission"
             "createMBeanServer";
  permission "javax.management.MBeanPermission"
             "-#-[-]", "queryNames";

Unfortunately, the original error remained and the error messages produced by the remote engine still contained the line

Task id 1588075306911:1378015012 Failed! Check remote policy file

So it looked like other permissions needed to be set but the corresponding exceptions were swallowed up somewhere in the code. As a last resort, which WORKED, I put the following into the remote_engine.policy file (and nothing else):

grant {
    permission java.security.AllPermission "", "";
    };

Make sure only trusted machines can connect to the machine with the remote engine at the port used by the remote engine. (This is advisable anyway.)

Cheers,
Eibe

On 28/04/2020, at 10:30 PM, Joel Ratsaby <[hidden email]> wrote:

 XYZ is just  like your  /Users/mjh/.

Here is my classpath on the remote engine machine:

java  -Xmx256m -XX:+UseSerialGC \
 -cp "/home/ratsaby/MY_HSQLDB/hsqldb.jar:/home/ratsaby/WEKA/jars/lib/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/*:/home/ratsaby/wekafiles/packages/wekaDeeplearning4j/lib/*:/home/ratsaby/WEKA/jars/*" \
 -Djava.security.policy=/home/ratsaby/WEKA/remote_engine/remote.policy \

I created the  WEKA jar from the sources (I downloaded weka 3.8.4). (Note again that the same classpath works well for standalone, namely, for running DL4j on Experimenter without RemoteEngine. the DL4j problem just arises when I use RemoteEngine).

Should I try it with weka-3-9?

I don't understand your conclusion. Did you find the reason? If so what do I need to add to the classpath of the Remoteengine? 




On Tue, 28 Apr 2020 at 04:33, Michael Hall <[hidden email]> wrote:


On Apr 27, 2020, at 12:48 PM, Joel Ratsaby <[hidden email]> wrote:

org.nd4j.linalg.factory.Nd4j

Based on this…

java -cp halfpipe.jar:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/lib/*:/Users/mjh//wekafiles/packages/wekaDeeplearning4j/*:/Applications/weka-3-9-3/weka.jar org.cmdline.cmds.showc org.nd4j.linalg.factory.Nd4j
showc: invoked with: org.nd4j.linalg.factory.Nd4j
java.lang.ExceptionInInitializerError
at org.apache.logging.log4j.util.StackLocatorUtil.<clinit>(StackLocatorUtil.java:28)
at org.apache.logging.slf4j.Log4jLoggerFactory.getContext(Log4jLoggerFactory.java:42)
at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:46)
at org.apache.logging.slf4j.Log4jLoggerFactory.getLogger(Log4jLoggerFactory.java:29)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:358)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:383)
at org.nd4j.linalg.factory.Nd4jBackend.<clinit>(Nd4jBackend.java:64)
at org.nd4j.linalg.factory.Nd4j.initContext(Nd4j.java:5152)
at org.nd4j.linalg.factory.Nd4j.<clinit>(Nd4j.java:182)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:340)
at org.cmdline.cmds.showc.main(showc.java:78)
Caused by: java.security.AccessControlException: access denied ("java.lang.RuntimePermission" "getStackWalkerWithClassReference")
at java.base/java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
at java.base/java.security.AccessController.checkPermission(AccessController.java:1036)
at java.base/java.lang.SecurityManager.checkPermission(SecurityManager.java:408)
at java.base/java.lang.StackWalker.checkPermission(StackWalker.java:424)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:365)
at java.base/java.lang.StackWalker.getInstance(StackWalker.java:336)
at org.apache.logging.log4j.util.StackLocator.<clinit>(StackLocator.java:28)
... 12 more

Since it seems to be loading correctly I think there is something not quite right about the class path. Whatever the XYZ is? 

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


--
Prof. Joel Ratsaby
Electrical and Electronics Engineering Dept.
Ariel University
ARIEL 40700
ISRAEL


_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html

_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html


--
Prof. Joel Ratsaby
Electrical and Electronics Engineering Dept.
Ariel University
ARIEL 40700
ISRAEL



_______________________________________________
Wekalist mailing list -- [hidden email]
Send posts to [hidden email]
To unsubscribe send an email to [hidden email]
To subscribe, unsubscribe, etc., visit https://list.waikato.ac.nz/postorius/lists/wekalist.list.waikato.ac.nz
List etiquette: http://www.cs.waikato.ac.nz/~ml/weka/mailinglist_etiquette.html