Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2992

ZKRMStateStore crashes due to session expiry

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      We recently saw the RM crash with the following stacktrace. On session expiry, we should gracefully transition to standby.

      2014-12-18 06:28:42,689 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type STATE_STORE_OP_FAILED. Cause: 
      org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired 
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) 
      at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:931) 
      at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:930) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:927) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1069) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1088) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:927) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:941) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.setDataWithRetries(ZKRMStateStore.java:958) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationAttemptStateInternal(ZKRMStateStore.java:687) 
      

        Attachments

        1. yarn-2992-1.patch
          1 kB
          Karthik Kambatla

          Activity

            People

            • Assignee:
              kasha Karthik Kambatla
              Reporter:
              kasha Karthik Kambatla
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: