Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2992

ZKRMStateStore crashes due to session expiry

    XMLWordPrintableJSON

Details

    • Reviewed

    Description

      We recently saw the RM crash with the following stacktrace. On session expiry, we should gracefully transition to standby.

      2014-12-18 06:28:42,689 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Received a org.apache.hadoop.yarn.server.resourcemanager.RMFatalEvent of type STATE_STORE_OP_FAILED. Cause: 
      org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired 
      at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) 
      at org.apache.zookeeper.ZooKeeper.multiInternal(ZooKeeper.java:931) 
      at org.apache.zookeeper.ZooKeeper.multi(ZooKeeper.java:911) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:930) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$4.run(ZKRMStateStore.java:927) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithCheck(ZKRMStateStore.java:1069) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore$ZKAction.runWithRetries(ZKRMStateStore.java:1088) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:927) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.doMultiWithRetries(ZKRMStateStore.java:941) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.setDataWithRetries(ZKRMStateStore.java:958) 
      at org.apache.hadoop.yarn.server.resourcemanager.recovery.ZKRMStateStore.updateApplicationAttemptStateInternal(ZKRMStateStore.java:687) 
      

      Attachments

        1. yarn-2992-1.patch
          1 kB
          Karthik Kambatla

        Activity

          People

            kasha Karthik Kambatla
            kasha Karthik Kambatla
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: