Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10722

Standby NN continuing as standby when active NN machine got shutdown.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Not A Problem
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: auto-failover, ha
    • Labels:
      None

      Description

      I have HA cluster with 3 ZK, 3 QJM.
      My Active NN machine got shutdown, but still my standby NN is standby only.
      It should be active

      ZKFC logs
      ========

      2014-06-19 13:39:30,810 INFO org.apache.hadoop.ha.NodeFencer: ====== Beginning Service Fencing Process... ======
      2014-06-19 13:39:30,810 INFO org.apache.hadoop.ha.NodeFencer: Trying method 1/1: org.apache.hadoop.ha.SshFenceByTcpPort(null)
      2014-06-19 13:39:30,811 INFO org.apache.hadoop.ha.SshFenceByTcpPort: Connecting to host-10-18-40-101...
      2014-06-19 13:39:30,811 INFO org.apache.hadoop.ha.SshFenceByTcpPort.jsch: Connecting to host-10-18-40-101 port 22
      2014-06-19 13:39:33,814 WARN org.apache.hadoop.ha.SshFenceByTcpPort: Unable to connect to host-10-18-40-101 as user myuser
      com.jcraft.jsch.JSchException: java.net.NoRouteToHostException: No route to host
      	at com.jcraft.jsch.Util.createSocket(Util.java:386)
      	at com.jcraft.jsch.Session.connect(Session.java:182)
      	at org.apache.hadoop.ha.SshFenceByTcpPort.tryFence(SshFenceByTcpPort.java:100)
      	at org.apache.hadoop.ha.NodeFencer.fence(NodeFencer.java:97)
      	at org.apache.hadoop.ha.ZKFailoverController.doFence(ZKFailoverController.java:521)
      	at org.apache.hadoop.ha.ZKFailoverController.fenceOldActive(ZKFailoverController.java:494)
      	at org.apache.hadoop.ha.ZKFailoverController.access$1100(ZKFailoverController.java:59)
      	at org.apache.hadoop.ha.ZKFailoverController$ElectorCallbacks.fenceOldActive(ZKFailoverController.java:837)
      	at org.apache.hadoop.ha.ActiveStandbyElector.fenceOldActive(ActiveStandbyElector.java:901)
      	at org.apache.hadoop.ha.ActiveStandbyElector.becomeActive(ActiveStandbyElector.java:800)
      	at org.apache.hadoop.ha.ActiveStandbyElector.processResult(ActiveStandbyElector.java:415)
      	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:596)
      	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)
      2014-06-19 13:39:33,814 WARN org.apache.hadoop.ha.NodeFencer: Fencing method org.apache.hadoop.ha.SshFenceByTcpPort(null) was unsuccessful.
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              surendrasingh Surendra Singh Lilhore
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: