Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-532

Fail create session with spark,yarn-client mode

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.6.0
    • 0.9.0
    • API, Interpreter, REPL, Server
    • None

    Description

      I'm using

      • spark2.3
      • livy 0.6.0 snapshot (current master, last commit: 0442eb01aa293d540fdf23ddd86b86c4278b83b9)
      • yarn client mode

      I tried add jar in create session params.

      ```
      18/11/05 08:17:16 WARN rsc.RSCClient: Client RPC channel closed unexpectedly.
      18/11/05 08:17:16 WARN rsc.RSCClient: Error stopping RPC.
      io.netty.util.concurrent.BlockingOperationException: DefaultChannelPromise@638f9c7b(uncancellable)
      at io.netty.util.concurrent.DefaultPromise.checkDeadLock(DefaultPromise.java:394)
      at io.netty.channel.DefaultChannelPromise.checkDeadLock(DefaultChannelPromise.java:157)
      at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:230)
      at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:129)
      at io.netty.channel.DefaultChannelPromise.await(DefaultChannelPromise.java:28)
      at io.netty.util.concurrent.DefaultPromise.sync(DefaultPromise.java:336)
      at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:117)
      at io.netty.channel.DefaultChannelPromise.sync(DefaultChannelPromise.java:28)
      at org.apache.livy.rsc.rpc.Rpc.close(Rpc.java:310)
      at org.apache.livy.rsc.RSCClient.stop(RSCClient.java:232)
      at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:129)
      at org.apache.livy.rsc.RSCClient$2$1.onSuccess(RSCClient.java:123)
      at org.apache.livy.rsc.Utils$2.operationComplete(Utils.java:108)
      at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:518)
      at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:492)
      at io.netty.util.concurrent.DefaultPromise.notifyListenersWithStackOverFlowProtection(DefaultPromise.java:431)
      at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:420)
      at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:108)
      at io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
      at io.netty.channel.AbstractChannel$CloseFuture.setClosed(AbstractChannel.java:995)
      at io.netty.channel.AbstractChannel$AbstractUnsafe.doClose0(AbstractChannel.java:621)
      at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:599)
      at io.netty.channel.AbstractChannel$AbstractUnsafe.close(AbstractChannel.java:543)
      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.closeOnRead(AbstractNioByteChannel.java:71)
      at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:158)
      at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:564)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:505)
      at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:419)
      at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:391)
      at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:112)
      at java.lang.Thread.run(Thread.java:748)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: Exception in thread "main" scala.reflect.internal.FatalError: object Predef does not have a member classOf
      18/11/05 08:17:16 INFO rsc.RSCClient: Failing pending job 3091ce65-e328-4c0f-9585-a861b90706d4 due to shutdown.
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.scala$reflect$internal$Definitions$DefinitionsClass$$fatalMissingSymbol(Definitions.scala:1186)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.getMember(Definitions.scala:1203)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass.getMemberMethod(Definitions.scala:1238)
      18/11/05 08:17:16 INFO interactive.InteractiveSession: Stopping InteractiveSession 2...
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf$lzycompute(Definitions.scala:1469)
      18/11/05 08:17:16 INFO interactive.InteractiveSession: Failed to ping RSC driver for session 2. Killing application.
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.Predef_classOf(Definitions.scala:1469)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.reflect.internal.Definitions$DefinitionsClass$RunDefinitions.isPredefClassOf(Definitions.scala:1459)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedIdent$2(Typers.scala:4885)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedIdentOrWildcard$1(Typers.scala:4908)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedInAnyMode$1(Typers.scala:5340)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5360)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5472)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedQualifier(Typers.scala:5480)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedPackageDef$1(Typers.scala:5012)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typedMemberDef$1(Typers.scala:5312)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed1(Typers.scala:5359)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.runTyper$1(Typers.scala:5396)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.scala$tools$nsc$typechecker$Typers$Typer$$typedInternal(Typers.scala:5423)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.body$2(Typers.scala:5370)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5374)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.ReplGlobal$$anon$1$$anon$2.typed(ReplGlobal.scala:36)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Typers$Typer.typed(Typers.scala:5448)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.apply(Analyzer.scala:102)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase$$anonfun$applyPhase$1.apply$mcV$sp(Global.scala:440)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase.withCurrentUnit(Global.scala:431)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$GlobalPhase.applyPhase(Global.scala:440)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:94)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3$$anonfun$run$1.apply(Analyzer.scala:93)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.collection.Iterator$class.foreach(Iterator.scala:893)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.typechecker.Analyzer$typerFactory$$anon$3.run(Analyzer.scala:93)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileUnitsInternal(Global.scala:1501)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileUnits(Global.scala:1486)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.Global$Run.compileSources(Global.scala:1481)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.compileSourcesKeepingRun(IMain.scala:435)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compileAndSaveRun(IMain.scala:855)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.compile(IMain.scala:813)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.bind(IMain.scala:675)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$bind$1.apply(SparkInterpreter.scala:133)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.tools.nsc.interpreter.IMain.beQuietDuring(IMain.scala:214)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter.bind(SparkInterpreter.scala:132)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.AbstractSparkInterpreter.postStart(AbstractSparkInterpreter.scala:72)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply$mcV$sp(SparkInterpreter.scala:89)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter$$anonfun$start$1.apply(SparkInterpreter.scala:63)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.AbstractSparkInterpreter.restoreContextClassLoader(AbstractSparkInterpreter.scala:340)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.SparkInterpreter.start(SparkInterpreter.scala:63)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:128)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at org.apache.livy.repl.Session$$anonfun$1.apply(Session.scala:122)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      18/11/05 08:17:16 INFO utils.LineBufferedStream: at java.lang.Thread.run(Thread.java:748)

      ```

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            kty1965 TaeYoung Kim
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated: