Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3443

Fix NPE when namenode transition to active during startup by adding checkNNStartup() in NameNodeRpcServer

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.1, 3.0.0-alpha1
    • Component/s: auto-failover, ha
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Start NN
      Let NN standby services be started.
      Before the editLogTailer is initialised start ZKFC and allow the activeservices start to proceed further.

      Here editLogTailer.catchupDuringFailover() will throw NPE.

      void startActiveServices() throws IOException {
      LOG.info("Starting services required for active state");
      writeLock();
      try {
      FSEditLog editLog = dir.fsImage.getEditLog();

      if (!editLog.isOpenForWrite())

      { // During startup, we're already open for write during initialization. editLog.initJournalsForWrite(); // May need to recover editLog.recoverUnclosedStreams(); LOG.info("Catching up to latest edits from old active before " + "taking over writer role in edits logs."); editLogTailer.catchupDuringFailover(); {noformat}

      2012-05-18 16:51:27,585 WARN org.apache.hadoop.ipc.Server: IPC Server Responder, call org.apache.hadoop.ha.HAServiceProtocol.getServiceStatus from XX.XX.XX.55:58003: output error
      2012-05-18 16:51:27,586 WARN org.apache.hadoop.ipc.Server: IPC Server handler 8 on 8020, call org.apache.hadoop.ha.HAServiceProtocol.transitionToActive from XX.XX.XX.55:58004: error: java.lang.NullPointerException
      java.lang.NullPointerException
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.startActiveServices(FSNamesystem.java:602)
      at org.apache.hadoop.hdfs.server.namenode.NameNode$NameNodeHAContext.startActiveServices(NameNode.java:1287)
      at org.apache.hadoop.hdfs.server.namenode.ha.ActiveState.enterState(ActiveState.java:61)
      at org.apache.hadoop.hdfs.server.namenode.ha.HAState.setStateInternal(HAState.java:63)
      at org.apache.hadoop.hdfs.server.namenode.ha.StandbyState.setState(StandbyState.java:49)
      at org.apache.hadoop.hdfs.server.namenode.NameNode.transitionToActive(NameNode.java:1219)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.transitionToActive(NameNodeRpcServer.java:978)
      at org.apache.hadoop.ha.protocolPB.HAServiceProtocolServerSideTranslatorPB.transitionToActive(HAServiceProtocolServerSideTranslatorPB.java:107)
      at org.apache.hadoop.ha.proto.HAServiceProtocolProtos$HAServiceProtocolService$2.callBlockingMethod(HAServiceProtocolProtos.java:3633)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:427)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:916)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1692)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1688)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1686)
      2012-05-18 16:51:27,586 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 8020 caught an exception
      java.nio.channels.ClosedChannelException
      at sun.nio.ch.SocketChannelImpl.ensureWriteOpen(SocketChannelImpl.java:133)
      at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:324)
      at org.apache.hadoop.ipc.Server.channelWrite(Server.java:2092)
      at org.apache.hadoop.ipc.Server.access$2000(Server.java:107)
      at org.apache.hadoop.ipc.Server$Responder.processResponse(Server.java:930)
      at org.apache.hadoop.ipc.Server$Responder.doRespond(Server.java:994)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1738)

      
      

        Attachments

        1. HDFS-3443_1.patch
          1 kB
          amith
        2. HDFS-3443_1.patch
          1 kB
          amith
        3. HDFS-3443-003.patch
          5 kB
          Vinayakumar B
        4. HDFS-3443-004.patch
          7 kB
          Vinayakumar B
        5. HDFS-3443-005.patch
          7 kB
          Vinayakumar B
        6. HDFS-3443-006.patch
          36 kB
          Vinayakumar B
        7. HDFS-3443-007.patch
          36 kB
          Vinayakumar B

          Issue Links

            Activity

              People

              • Assignee:
                vinayrpet Vinayakumar B
                Reporter:
                suja suja s
              • Votes:
                0 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: