Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-776

CI: PySpark tests failure on PR208

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.6.0
    • Component/s: CI-infra
    • Labels:

      Description

      zeppelin-server tests in ZeppelinSparkClusterTest consistently failing on pySpark part

      -----------------

      TODOs:

      • General
      • PR 208

      -------------------

      Example https://travis-ci.org/bzz/incubator-zeppelin/builds/119447025

      Failed tests: 
        ZeppelinSparkClusterTest.pySparkDepLoaderTest:167->getSparkVersionNumber:220 expected:<FINISHED> but was:<ERROR>
        ZeppelinSparkClusterTest.pySparkAutoConvertOptionTest:112->getSparkVersionNumber:220 expected:<FINISHED> but was:<ERROR>
        ZeppelinSparkClusterTest.basicRDDTransformationAndActionTest:81 expected:<FINISHED> but was:<ERROR>
        ZeppelinSparkClusterTest.pySparkTest:90->getSparkVersionNumber:220 expected:<FINISHED> but was:<ERROR>
        ZeppelinSparkClusterTest.zRunTest:152 expected:<FINISHED> but was:<ERROR>
      
      Tests run: 35, Failures: 5, Errors: 0, Skipped: 0
      
      [INFO] ------------------------------------------------------------------------
      [INFO] Reactor Summary:
      [INFO] 
      [INFO] Zeppelin: Interpreter .............................. SUCCESS [ 11.901 s]
      [INFO] Zeppelin: Zengine .................................. SUCCESS [  3.691 s]
      [INFO] Zeppelin: Display system apis ...................... SUCCESS [  2.158 s]
      [INFO] Zeppelin: Spark dependencies ....................... SUCCESS [ 30.747 s]
      [INFO] Zeppelin: Spark .................................... SUCCESS [  2.912 s]
      [INFO] Zeppelin: Server ................................... FAILURE [01:52 min]
      [INFO] Zeppelin: R Interpreter ............................ SKIPPED
      [INFO] ------------------------------------------------------------------------
      [INFO] BUILD FAILURE
      

      Two things are suspicious in the logs:
      a) IndexOutOfBoundsException (but may be harmless)
      b) Spark Interpreter is not able to communicate to SparkMaster

      A. IndexOutOfBoundsException

      06:28:14,435 ERROR org.apache.zeppelin.rest.NotebookRestApi:405 - Exception in NotebookRestApi while moveParagraph 
      java.lang.IndexOutOfBoundsException: paragraph size is 2 , index is 10
      	at org.apache.zeppelin.notebook.Note.moveParagraph(Note.java:274)
      	at org.apache.zeppelin.rest.NotebookRestApi.moveParagraph(NotebookRestApi.java:399)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      

      B. Spark Interpreter is not able to communicate to SparkMaster

      16/03/30 06:28:24 INFO SparkContext: Added JAR file:/home/travis/build/bzz/incubator-zeppelin/interpreter/spark/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar at http://10.128.0.211:47567/jars/zeppelin-spark-0.6.0-incubating-SNAPSHOT.jar with timestamp 1459319304855
      16/03/30 06:28:24 INFO FairSchedulableBuilder: Created default pool default, schedulingMode: FIFO, minShare: 0, weight: 1
      16/03/30 06:28:24 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.
      16/03/30 06:28:24 INFO AppClient$ClientEndpoint: Connecting to master spark://testing-gce-3dcf1a9b-ea54-400d-b356-693b37a2edd4.c.travis-ci-prod-5.internal:7071...
      16/03/30 06:28:44 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[appclient-registration-retry-thread,5,main]
      java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.FutureTask@f431bb4 rejected from java.util.concurrent.ThreadPoolExecutor@7ecfce3f[Running, pool size = 1, active threads = 1, queued tasks = 0, completed tasks = 0]
      	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
      	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
      

        Attachments

          Activity

            People

            • Assignee:
              bzz Alexander Bezzubov
              Reporter:
              bzz Alexander Bezzubov
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: