Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1159

Livy interpreter gets "404 not found" error

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.1, 0.7.0
    • Component/s: zeppelin-interpreter
    • Labels:
      None

      Description

      Livy SparkR gets "404 not found" error

      Scenario:

      • Start a notebook and Run below sparkR paragraph.
        %sh
        hdfs dfs -copyFromLocal examples/src/main/resources/people.json /tmp
      %livy.sparkr
      sc <- sparkR.init(sparkPackages="com.databricks:spark-csv_2.11:1.0.3")
      sqlContext <- sparkRSQL.init(sc)
      people <- read.df(sqlContext, "/tmp/people.json", "json") 
      head(people)
      printSchema(people)
      write.df(people, path="people.parquet", source="parquet", mode="overwrite")
      ============================output ==================================
      Re-using existing Spark Context. Please stop SparkR with sparkR.stop() or restart R to create a new Spark Context
        age    name
      1  NA Michael
      2  30    Andy
      3  19  Justin
      root
       |-- age: long (nullable = true)
       |-- name: string (nullable = true)
      NULL
      Attaching package: ‘SparkR’
      The following objects are masked from ‘package:stats’:
          cov, filter, lag, na.omit, predict, sd, var
      The following objects are masked from ‘package:base’:
          colnames, colnames<-, endsWith, intersect, rank, rbind, sample,
          startsWith, subset, summary, table, transform
      • let the livy session expire.
      • run the same paragraph again.
        %livy.sparkR fails to execute with "404 Not Found"
      livy interpreter log
       INFO [2016-07-12 22:59:35,395] ({pool-2-thread-9} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1468364375373 finished by scheduler org.apache.zeppelin.livy.LivySparkRInterpreter1028865681
       INFO [2016-07-12 22:59:37,620] ({pool-2-thread-5} SchedulerFactory.java[jobStarted]:131) - Job remoteInterpretJob_1468364377620 started by scheduler org.apache.zeppelin.livy.LivySparkRInterpreter1028865681
      ERROR [2016-07-12 22:59:37,633] ({pool-2-thread-5} LivySparkRInterpreter.java[interpret]:81) - Exception in LivySparkRInterpreter while interpret
      org.springframework.web.client.HttpClientErrorException: 404 Not Found
              at org.springframework.web.client.DefaultResponseErrorHandler.handleError(DefaultResponseErrorHandler.java:91)
              at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:667)
              at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:620)
              at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580)
              at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498)
              at org.apache.zeppelin.livy.LivyHelper.executeHTTP(LivyHelper.java:354)
              at org.apache.zeppelin.livy.LivyHelper.executeCommand(LivyHelper.java:299)
              at org.apache.zeppelin.livy.LivyHelper.interpret(LivyHelper.java:240)
              at org.apache.zeppelin.livy.LivySparkRInterpreter.interpret(LivySparkRInterpreter.java:79)
              at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
              at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
              at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
       INFO [2016-07-12 22:59:37,637] ({pool-2-thread-5} SchedulerFactory.java[jobFinished]:137) - Job remoteInterpretJob_1468364377620 finished by scheduler org.apache.zeppelin.livy.LivySparkRInterpreter1028865681

      This issue looks like Livy interpreter specific.
      The interpreter works fine after restarting livy interpreter.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                yeshavora Yesha Vora
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: