Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-196

Livy not loading yarn-site temporarily after being restarted

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.2
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      After the livy server is restarted, and the first spark job is submitted through Livy - we see in the livy server logs that it is trying to connect to the RM at default address 0.0.0.0:8032 even though RM webaddress in yarn-site.xml is a non-default

      6/07/15 19:45:45 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:46 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:47 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:48 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:49 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:50 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:51 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:52 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:53 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      16/07/15 19:45:54 INFO Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
      Exception in thread "session app state monitor thread" java.net.ConnectException: Call From XX.XX.XX.XX to 0.0.0.0:8032 failed on connection exception: java.net.ConnectException: Connection refused; For more details see:  http://wiki.apache.org/hadoop/ConnectionRefused
      	at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
      	at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:801)
      	at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:732)
      	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1555)
      	at org.apache.hadoop.ipc.Client.call(Client.java:1495)
      	at org.apache.hadoop.ipc.Client.call(Client.java:1395)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:233)
      	at com.sun.proxy.$Proxy8.getApplicationReport(Unknown Source)
      	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getApplicationReport(ApplicationClientProtocolPBClientImpl.java:191)
      	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:606)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:256)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:104)
      	at com.sun.proxy.$Proxy9.getApplicationReport(Unknown Source)
      	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getApplicationReport(YarnClientImpl.java:431)
      	at com.cloudera.livy.sessions.SessionManager$$anonfun$2.apply(SessionManager.scala:108)
      	at com.cloudera.livy.sessions.SessionManager$$anonfun$2.apply(SessionManager.scala:105)
      	at scala.collection.immutable.List.foreach(List.scala:318)
      	at com.cloudera.livy.sessions.SessionManager.checkAppState(SessionManager.scala:105)
      	at com.cloudera.livy.sessions.SessionManager$SessionAppStateMonitor.run(SessionManager.scala:142)
      
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              kbadani Kshitij Badani
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: