Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14081

hdfs dfsadmin -metasave metasave_test results NPE

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.2.1
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: hdfs
    • Labels:
      None
    • Target Version/s:

      Description

      Race condition is encountered while adding Block to postponedMisreplicatedBlocks which in turn tried to retrieve Block from BlockManager in which it may not be present.

      This happens in HA, metasave in first NN succeeded but failed in second NN, StackTrace showing NPE is as follows:

      2018-07-12 21:39:09,783 WARN org.apache.hadoop.ipc.Server: IPC Server handler 24 on 8020, call Call#1 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.metaSave from 172.26.9.163:602342018-07-12 21:39:09,783 WARN org.apache.hadoop.ipc.Server: IPC Server handler 24 on 8020, call Call#1 Retry#0 org.apache.hadoop.hdfs.protocol.ClientProtocol.metaSave from 172.26.9.163:60234java.lang.NullPointerException at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseSourceDatanodes(BlockManager.java:2175) at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.dumpBlockMeta(BlockManager.java:830) at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.metaSave(BlockManager.java:762) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.metaSave(FSNamesystem.java:1782) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.metaSave(FSNamesystem.java:1766) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.metaSave(NameNodeRpcServer.java:1320) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.metaSave(ClientNamenodeProtocolServerSideTranslatorPB.java:928) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869) at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1685) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675) 

        Attachments

        1. HDFS-14081.010.patch
          10 kB
          Shweta
        2. HDFS-14081.009.patch
          9 kB
          Shweta
        3. HDFS-14081.008.patch
          7 kB
          Shweta
        4. HDFS-14081.007.patch
          7 kB
          Shweta
        5. HDFS-14081.006.patch
          7 kB
          Shweta
        6. HDFS-14081.005.patch
          5 kB
          Shweta
        7. HDFS-14081.004.patch
          4 kB
          Shweta
        8. HDFS-14081.003.patch
          4 kB
          Shweta
        9. HDFS-14081.002.patch
          4 kB
          Shweta
        10. HDFS-14081.001.patch
          4 kB
          Shweta

          Activity

            People

            • Assignee:
              shwetayakkali Shweta
              Reporter:
              shwetayakkali Shweta
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: