Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-3991

Fail to bootstrap PySpark in yarn client mode

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: pySpark, python-interpreter
    • Labels:
      None
    • Environment:

      Description

      Hi,

      PySpark fails with following error in Zeppelin 0.9.0:

      ERROR [2019-02-06 22:23:40,599] ({FIFOScheduler-Worker-1} Job.java[run]:174) - Job failed
      org.apache.zeppelin.interpreter.InterpreterException: Fail to bootstrap pyspark
      at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:124)
      at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
      at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:593)
      at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:502)
      at org.apache.zeppelin.scheduler.Job.run(Job.java:172)
      at org.apache.zeppelin.scheduler.AbstractScheduler.runJob(AbstractScheduler.java:121)
      at org.apache.zeppelin.scheduler.FIFOScheduler.lambda$runJobInScheduler$0(FIFOScheduler.java:39)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.io.IOException: Fail to run bootstrap script: python/zeppelin_pyspark.py
      at org.apache.zeppelin.python.PythonInterpreter.bootstrapInterpreter(PythonInterpreter.java:581)
      at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:122)
      ... 9 more
      

      Full logs of YARN:

      https://gist.github.com/maziyarpanahi/3380e230246271217a2feb4512f5d665

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              maziyar Maziyar PANAHI
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: