[ZEPPELIN-783] CI: spark download fails the CI on master

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Component/s: CI-infra
• Labels: CI flaky-test
• Assignee: Alexander Bezzubov
• Reporter: Alexander Bezzubov
• Created: 01/Apr/16 02:25
• Updated: 11/Apr/16 12:02
• Resolved: 11/Apr/16 12:02

Description

CI is red on master now for 2 last commits.

Both are the cases when Spark download fails on CI machine, although the symptoms looks similar with ZEPPELIN-776 - those are different issues.

TODO:

• fail the build if download retries have failed

---------------

https://travis-ci.org/apache/incubator-zeppelin/jobs/119830010

Using worker: worker-linux-docker-0a828f8e.prod.travis-ci.org:travis-linux-10

[33;1mThis job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.

The command "./testing/downloadSpark.sh 1.6.1 2.3" failed. Retrying, 3 of 3.

15:38:19,236  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:119 - Run interpreter process [..//bin/interpreter.sh, -d, /home/travis/build/apache/incubator-zeppelin/zeppelin-server/../interpreter/spark, -p, 38590, -l, ..//local-repo/2BGB3JVKH]
..//bin/interpreter.sh: line 151: /home/travis/build/apache/incubator-zeppelin/run/zeppelin-interpreter-spark-travis-testing-worker-linux-docker-0a828f8e-3372-linux-10.pid: No such file or directory
Error: Could not find or load main class org.apache.spark.launcher.Main
15:38:19,346  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:247 - Interpreter process exited 0

https://travis-ci.org/apache/incubator-zeppelin/jobs/119884848

Using worker: worker-linux-docker-bbc4c63f.prod.travis-ci.org:travis-linux-2

33;1mThis job is running on container-based infrastructure, which does not allow use of 'sudo', setuid and setguid executables.

18:58:54,371  INFO org.apache.zeppelin.notebook.Paragraph:221 - run paragraph 20160331-185854_1215648458 using spark org.apache.zeppelin.interpreter.LazyOpenInterpreter@512eb371
18:58:54,386  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:119 - Run interpreter process [..//bin/interpreter.sh, -d, /home/travis/build/apache/incubator-zeppelin/zeppelin-server/../interpreter/spark, -p, 57886, -l, ..//local-repo/2BEPXYAQR]
Log dir doesn't exist, create /home/travis/build/apache/incubator-zeppelin/logs
..//bin/interpreter.sh: line 151: /home/travis/build/apache/incubator-zeppelin/run/zeppelin-interpreter-spark-travis-testing-worker-linux-docker-bbc4c63f-3487-linux-2.pid: No such file or directory
Failed to find Spark assembly in /home/travis/build/apache/incubator-zeppelin/spark-1.5.2-bin-hadoop2.3/lib.
You need to build Spark before running this program.
18:58:54,418  INFO org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess:247 - Interpreter process exited 0

Work has already been done by felixcheung at #727 to metigate this issue, but it neens some more improvements.