Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10715

NPE when applying AvailableSpaceBlockPlacementPolicy

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: namenode
    • Labels:
      None
    • Environment:

      cdh5.8.0

    • Hadoop Flags:
      Reviewed

      Description

      As HDFS-8131 introduced an AvailableSpaceBlockPlacementPolicy, but In some cases, it caused NPE.

      Here are my namenode daemon logs :

      2016-08-02 13:05:03,271 WARN org.apache.hadoop.ipc.Server: IPC Server handler 13 on 8020, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 10.132.89.79:14001 Call#56 Retry#0
      java.lang.NullPointerException
      at org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceBlockPlacementPolicy.compareDataNode(AvailableSpaceBlockPlacementPolicy.java:95)
      at org.apache.hadoop.hdfs.server.blockmanagement.AvailableSpaceBlockPlacementPolicy.chooseDataNode(AvailableSpaceBlockPlacementPolicy.java:80)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:691)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseRandom(BlockPlacementPolicyDefault.java:665)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseLocalRack(BlockPlacementPolicyDefault.java:572)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTargetInOrder(BlockPlacementPolicyDefault.java:457)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:367)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:242)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:114)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault.chooseTarget(BlockPlacementPolicyDefault.java:130)
      at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1606)
      at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3315)
      at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:679)
      at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.addBlock(AuthorizationProviderProxyClientProtocol.java:214)
      at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:489)
      at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
      at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1073)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:422)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)

      I reviewed the source code, and found the bug in method chooseDataNode. clusterMap.chooseRandom may return null, which cannot compare using equals a.equals(b) method.

      Though this exception can be caught, and then retry another call. I think this bug should be fixed.

      1. HDFS-10715.001.patch
        1 kB
        Guangbin Zhu
      2. HDFS-10715.002.patch
        3 kB
        Guangbin Zhu
      3. HDFS-10715.003.patch
        3 kB
        Guangbin Zhu
      4. HDFS-10715.004.patch
        3 kB
        Guangbin Zhu

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 11s trunk passed
        +1 compile 0m 47s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 0m 53s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 45s trunk passed
        +1 javadoc 0m 56s trunk passed
        +1 mvninstall 0m 48s the patch passed
        +1 compile 0m 44s the patch passed
        +1 javac 0m 44s the patch passed
        +1 checkstyle 0m 22s the patch passed
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 45s the patch passed
        +1 javadoc 0m 54s the patch passed
        -1 unit 57m 34s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        76m 59s



        Reason Tests
        Failed junit tests hadoop.hdfs.TestFileCreationDelete



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821530/HDFS-10715.001.patch
        JIRA Issue HDFS-10715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 785be055fad4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 7fc70c6
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16290/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16290/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16290/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 11s trunk passed +1 compile 0m 47s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 53s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 45s trunk passed +1 javadoc 0m 56s trunk passed +1 mvninstall 0m 48s the patch passed +1 compile 0m 44s the patch passed +1 javac 0m 44s the patch passed +1 checkstyle 0m 22s the patch passed +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 45s the patch passed +1 javadoc 0m 54s the patch passed -1 unit 57m 34s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 76m 59s Reason Tests Failed junit tests hadoop.hdfs.TestFileCreationDelete Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821530/HDFS-10715.001.patch JIRA Issue HDFS-10715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 785be055fad4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7fc70c6 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16290/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16290/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16290/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        UT added

        Show
        zhuguangbin86 Guangbin Zhu added a comment - UT added
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 47s trunk passed
        +1 compile 0m 45s trunk passed
        +1 checkstyle 0m 25s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 41s trunk passed
        +1 javadoc 0m 54s trunk passed
        +1 mvninstall 0m 44s the patch passed
        +1 compile 0m 40s the patch passed
        +1 javac 0m 40s the patch passed
        -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 30 unchanged - 0 fixed = 34 total (was 30)
        +1 mvnsite 0m 49s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 53s the patch passed
        -1 unit 62m 13s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 19s The patch does not generate ASF License warnings.
        81m 3s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID
          hadoop.hdfs.TestDecommissionWithStriped



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821762/HDFS-10715.002.patch
        JIRA Issue HDFS-10715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5c7715300666 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d28c2d9
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16301/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16301/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16301/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16301/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 47s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 25s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 41s trunk passed +1 javadoc 0m 54s trunk passed +1 mvninstall 0m 44s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed -0 checkstyle 0m 23s hadoop-hdfs-project/hadoop-hdfs: The patch generated 4 new + 30 unchanged - 0 fixed = 34 total (was 30) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 53s the patch passed -1 unit 62m 13s hadoop-hdfs in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 81m 3s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.TestDecommissionWithStriped Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821762/HDFS-10715.002.patch JIRA Issue HDFS-10715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5c7715300666 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d28c2d9 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16301/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16301/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16301/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16301/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Mostly looks good to me. Some minor comments:
        1.

              BlockPlacementPolicy placementPolicy = namenode.getNamesystem().getBlockManager().getBlockPlacementPolicy();
        

        Can we reuse the static variable placementPolicy instead of local variable?
        2. Would you fix the checkstyle warnings?
        3.

              for (Node node: dataNodes) {
                allNodes.add(node);
              }
        

        can be replaced with Collections.addAll(allNodes, dataNodes);.

        Show
        ajisakaa Akira Ajisaka added a comment - Mostly looks good to me. Some minor comments: 1. BlockPlacementPolicy placementPolicy = namenode.getNamesystem().getBlockManager().getBlockPlacementPolicy(); Can we reuse the static variable placementPolicy instead of local variable? 2. Would you fix the checkstyle warnings? 3. for (Node node: dataNodes) { allNodes.add(node); } can be replaced with Collections.addAll(allNodes, dataNodes); .
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        Thanks Akira, I'll fix it and submit another patch.

        Show
        zhuguangbin86 Guangbin Zhu added a comment - Thanks Akira, I'll fix it and submit another patch.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        Thanks Akira, I'll fix it and submit another patch.

        Show
        zhuguangbin86 Guangbin Zhu added a comment - Thanks Akira, I'll fix it and submit another patch.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        code refactoring

        Show
        zhuguangbin86 Guangbin Zhu added a comment - code refactoring
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 8s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 52s trunk passed
        +1 compile 0m 46s trunk passed
        +1 checkstyle 0m 28s trunk passed
        +1 mvnsite 0m 56s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 47s trunk passed
        +1 javadoc 0m 58s trunk passed
        +1 mvninstall 0m 51s the patch passed
        +1 compile 0m 45s the patch passed
        +1 javac 0m 45s the patch passed
        -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 30 unchanged - 0 fixed = 32 total (was 30)
        +1 mvnsite 0m 53s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 2m 2s the patch passed
        +1 javadoc 1m 0s the patch passed
        -1 unit 65m 5s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        86m 12s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestEditLog



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821785/HDFS-10715.003.patch
        JIRA Issue HDFS-10715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 0abc293cddde 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d848184
        Default Java 1.8.0_101
        findbugs v3.0.0
        checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16304/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16304/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16304/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16304/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 52s trunk passed +1 compile 0m 46s trunk passed +1 checkstyle 0m 28s trunk passed +1 mvnsite 0m 56s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 47s trunk passed +1 javadoc 0m 58s trunk passed +1 mvninstall 0m 51s the patch passed +1 compile 0m 45s the patch passed +1 javac 0m 45s the patch passed -0 checkstyle 0m 25s hadoop-hdfs-project/hadoop-hdfs: The patch generated 2 new + 30 unchanged - 0 fixed = 32 total (was 30) +1 mvnsite 0m 53s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 2m 2s the patch passed +1 javadoc 1m 0s the patch passed -1 unit 65m 5s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 86m 12s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestEditLog Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821785/HDFS-10715.003.patch JIRA Issue HDFS-10715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 0abc293cddde 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d848184 Default Java 1.8.0_101 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/16304/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/16304/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16304/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16304/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        fix checkstyle

        Show
        zhuguangbin86 Guangbin Zhu added a comment - fix checkstyle
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 54s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 7m 12s trunk passed
        +1 compile 0m 44s trunk passed
        +1 checkstyle 0m 26s trunk passed
        +1 mvnsite 0m 51s trunk passed
        +1 mvneclipse 0m 12s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 0m 55s trunk passed
        +1 mvninstall 0m 47s the patch passed
        +1 compile 0m 43s the patch passed
        +1 javac 0m 43s the patch passed
        +1 checkstyle 0m 23s the patch passed
        +1 mvnsite 0m 48s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 46s the patch passed
        +1 javadoc 0m 52s the patch passed
        -1 unit 58m 10s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        78m 8s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821800/HDFS-10715.004.patch
        JIRA Issue HDFS-10715
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 55a5fae978a0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / d848184
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/16305/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16305/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16305/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 54s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 12s trunk passed +1 compile 0m 44s trunk passed +1 checkstyle 0m 26s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 55s trunk passed +1 mvninstall 0m 47s the patch passed +1 compile 0m 43s the patch passed +1 javac 0m 43s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 0m 48s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 52s the patch passed -1 unit 58m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 78m 8s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12821800/HDFS-10715.004.patch JIRA Issue HDFS-10715 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 55a5fae978a0 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d848184 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/16305/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/16305/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/16305/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kihwal Kihwal Lee added a comment -

        Guangbin Zhu, I've added you as a contributor and assigned this jira to you.

        Show
        kihwal Kihwal Lee added a comment - Guangbin Zhu , I've added you as a contributor and assigned this jira to you.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment - - edited

        Many thanks Kihwal Lee. I found one failed test

        hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics

        , but it is unrelated with this issue.

        Should I commit this patch ?

        Show
        zhuguangbin86 Guangbin Zhu added a comment - - edited Many thanks Kihwal Lee . I found one failed test hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics , but it is unrelated with this issue. Should I commit this patch ?
        Hide
        ajisakaa Akira Ajisaka added a comment -

        LGTM, +1. Thanks Guangbin Zhu for the update.

        Show
        ajisakaa Akira Ajisaka added a comment - LGTM, +1. Thanks Guangbin Zhu for the update.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        commited, thanks all for reviewing

        Show
        zhuguangbin86 Guangbin Zhu added a comment - commited, thanks all for reviewing
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Hi Guangbin Zhu, where did you committed? An issue is resolved and fix/version is set only when a patch is committed to the ASF git repository.

        Show
        ajisakaa Akira Ajisaka added a comment - Hi Guangbin Zhu , where did you committed? An issue is resolved and fix/version is set only when a patch is committed to the ASF git repository.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        Sorry Akira Ajisaka, I thought I click the "Resolve Issue" button , the patch will be commited automatically. Sorry for my mistake.

        Show
        zhuguangbin86 Guangbin Zhu added a comment - Sorry Akira Ajisaka , I thought I click the "Resolve Issue" button , the patch will be commited automatically. Sorry for my mistake.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        Would anybody else review this jira ?

        Show
        zhuguangbin86 Guangbin Zhu added a comment - Would anybody else review this jira ?
        Hide
        kihwal Kihwal Lee added a comment -

        Guangbin Zhu, please read http://wiki.apache.org/hadoop/HowToContribute .
        Since a committer(Akira) has +1'ed your patch, it won't need any more reviews unless someone has further concerns. Committing is done by only committers.

        Show
        kihwal Kihwal Lee added a comment - Guangbin Zhu , please read http://wiki.apache.org/hadoop/HowToContribute . Since a committer(Akira) has +1'ed your patch, it won't need any more reviews unless someone has further concerns. Committing is done by only committers.
        Hide
        zhuguangbin86 Guangbin Zhu added a comment -

        Thanks Kihwal Lee, Thank you for your patient.

        Show
        zhuguangbin86 Guangbin Zhu added a comment - Thanks Kihwal Lee , Thank you for your patient.
        Hide
        ajisakaa Akira Ajisaka added a comment -

        Committed this to trunk, branch-2, and branch-2.8. Thanks Guangbin Zhu for the contribution!

        Show
        ajisakaa Akira Ajisaka added a comment - Committed this to trunk, branch-2, and branch-2.8. Thanks Guangbin Zhu for the contribution!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #10221 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10221/)
        HDFS-10715. NPE when applying AvailableSpaceBlockPlacementPolicy. (aajisaka: rev ef432579a7763cc0e482fe049027c6e5325eb034)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestAvailableSpaceBlockPlacementPolicy.java
        • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #10221 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10221/ ) HDFS-10715 . NPE when applying AvailableSpaceBlockPlacementPolicy. (aajisaka: rev ef432579a7763cc0e482fe049027c6e5325eb034) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/blockmanagement/TestAvailableSpaceBlockPlacementPolicy.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/AvailableSpaceBlockPlacementPolicy.java

          People

          • Assignee:
            zhuguangbin86 Guangbin Zhu
            Reporter:
            zhuguangbin86 Guangbin Zhu
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development