[ZEPPELIN-302] Spark interpreter settings changes are not propagated to interpreter OS process args

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Component/s: Core, Interpreters
• Labels: interpreter-process

Environment

Linux ip-10-228-34-138 3.4.62-53.42.amzn1.x86_64 #1 SMP Fri Sep 20 07:23:24 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
Java HotSpot(TM) 64-Bit Server VM (build 23.21-b01, mixed mode)

Installed and build with:
git clone https://github.com/apache/incubator-zeppelin.git zeppelin
mvn clean package -Pspark-1.4 -Dhadoop.version=2.6.0 -Phadoop-2.6 -Pyarn -DskipTests

No changes in zeppelin/conf directory

• Assignee: Unassigned
• Reporter: Eugene Sapozhnikov
• Created: 13/Sep/15 13:23
• Updated: 05/Nov/15 12:06

Description

Steps to reproduce:

1. Go to interpreters page, edit spark item
2. Edit property 'args' to 'arg1 arg2'
3. Add property 'foo.bar' with value '11'
4. Restart zeppelin daemon, just to make sure all gets fetched
5. Create notebook, print 'println("11")' in a paragraph, execute it
6. Find spark interpreter item in OS processes: ps aux | grep spark
7. Expect to see 'arg1 arg2' and 'foo.bar' in this line. (FAIL)

I have this line:
zeppelin 2648 92.8 8.2 2401240 318784 pts/0 Sl 13:13 0:22 java -Dfile.encoding=UTF-8 -Xmx1024m -XX:MaxPermSize=512m -Dfile.encoding=UTF-8 -Xmx1024m -XX:MaxPermSize=512m -Dzeppelin.log.file=/home/zeppelin/zeppelin-test/logs/zeppelin-interpreter-spark-zeppelin-ip-10-228-34-138.log -cp :/home/zeppelin/zeppelin-test/interpreter/spark/dep/:/home/zeppelin/zeppelin-test/interpreter/spark/:/home/zeppelin/zeppelin-test/zeppelin-interpreter/target/lib/:/home/zeppelin/zeppelin-test/zeppelin-server/target/lib/:/home/zeppelin/zeppelin-test/zeppelin-zengine/target/lib/:/home/zeppelin/zeppelin-test/zeppelin-interpreter/target/lib/:/home/zeppelin/zeppelin-test/*::/home/zeppelin/zeppelin-test/conf:/home/zeppelin/zeppelin-test/zeppelin-interpreter/target/classes:/home/zeppelin/zeppelin-test/zeppelin-zengine/target/classes:/home/zeppelin/zeppelin-test/zeppelin-server/target/classes:/home/zeppelin/zeppelin-test/conf:/home/zeppelin/zeppelin-test/conf:/home/zeppelin/zeppelin-test/zeppelin-interpreter/target/classes org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer 36933

Maybe my assumption is not correct and args should not change. I tried to add spark.driver.extraClassPath property and I don't see any change in behavior with or without it (class still not found).