Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11135

The tests in TestBalancer run fails due to NPE

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha2
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I am making test for HDFS-11051, I found a serious issue: All the tests in TestBalancer fails except TestBalancer#testBalancerWithKeytabs. I saw there was NPE threw in the method TestBalancer#destroy. The stack infos:

      java.lang.NullPointerException
      	at org.apache.hadoop.fs.FileUtil.fullyDeleteContents(FileUtil.java:242)
      	at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:164)
      	at org.apache.hadoop.fs.FileUtil.fullyDelete(FileUtil.java:135)
      	at org.apache.hadoop.hdfs.server.balancer.TestBalancer.destroy(TestBalancer.java:274)
      

      The related codes:

        @AfterClass
        public static void destroy() throws Exception {
          if (kdc != null) {
            kdc.stop();
          }
          FileUtil.fullyDelete(baseDir);
            KeyStoreTestUtil.cleanupSSLConfig(keystoresDir, sslConfDir);
        }
      

      It seems that the deleted dir baseDir is only used in TestBalancer#testBalancerWithKeytabs, but here we don't do a check and then it throws the NPE. Finally cause the other tests fails. Since it will run the test TestBalancer#testBalancerWithKeytabs and initialize the dir if we do the whole test TesrBalancer, and it will runs good. However, it will fails if we want to run a single test in TestBalancer.

        Issue Links

          Activity

          Hide
          linyiqun Yiqun Lin added a comment -

          Attach a simple patch to make a quick fix. Thanks for the review.

          Show
          linyiqun Yiqun Lin added a comment - Attach a simple patch to make a quick fix. Thanks for the review.
          Hide
          brahmareddy Brahma Reddy Battula added a comment - - edited

          Yiqun Lin Nice Catch, it was introduced by HDFS-11042.
          Remaining test classes should be ok(which are updated from HDFS-11042) and can you also confirm once..?
          Patch LGTM, +1.Pending for jenkins.

          Show
          brahmareddy Brahma Reddy Battula added a comment - - edited Yiqun Lin Nice Catch, it was introduced by HDFS-11042 . Remaining test classes should be ok(which are updated from HDFS-11042 ) and can you also confirm once..? Patch LGTM, +1.Pending for jenkins.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          linking the broken jira.

          Show
          brahmareddy Brahma Reddy Battula added a comment - linking the broken jira.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 6s trunk passed
          +1 compile 0m 45s trunk passed
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 1m 42s trunk passed
          +1 javadoc 0m 39s trunk passed
          +1 mvninstall 0m 45s the patch passed
          +1 compile 0m 41s the patch passed
          +1 javac 0m 41s the patch passed
          +1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 206 unchanged - 1 fixed = 206 total (was 207)
          +1 mvnsite 0m 49s the patch passed
          +1 mvneclipse 0m 9s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 findbugs 1m 46s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 53m 36s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          72m 18s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeLifeline



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:a9ad5d6
          JIRA Issue HDFS-11135
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838747/HDFS-11135.001.patch
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 425c481e21c3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 79448d4
          Default Java 1.8.0_101
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/17551/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17551/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17551/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 6s trunk passed +1 compile 0m 45s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 0m 39s trunk passed +1 mvninstall 0m 45s the patch passed +1 compile 0m 41s the patch passed +1 javac 0m 41s the patch passed +1 checkstyle 0m 26s hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 206 unchanged - 1 fixed = 206 total (was 207) +1 mvnsite 0m 49s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 46s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 53m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 72m 18s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeLifeline Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11135 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12838747/HDFS-11135.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 425c481e21c3 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 79448d4 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17551/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17551/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17551/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          linyiqun Yiqun Lin added a comment - - edited

          Thanks Brahma Reddy Battula for the quick review.

          and can you also confirm once..?

          I have made a confirm and did test in my local. Remaining test classes seems ok.

          Show
          linyiqun Yiqun Lin added a comment - - edited Thanks Brahma Reddy Battula for the quick review. and can you also confirm once..? I have made a confirm and did test in my local. Remaining test classes seems ok.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Yiqun Lin thanks for confirmation. Test failure is Unrelated and raised HDFS-11136 to track.

          Committed to trunk,branch-2 and branch-2.8.

          Show
          brahmareddy Brahma Reddy Battula added a comment - Yiqun Lin thanks for confirmation. Test failure is Unrelated and raised HDFS-11136 to track. Committed to trunk,branch-2 and branch-2.8.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10831 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10831/)
          HDFS-11135. The tests in TestBalancer run fails due to NPE. Contributed (brahma: rev 6efb8c9c6018ec688eeba8c61c220009ec350a44)

          • (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10831 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10831/ ) HDFS-11135 . The tests in TestBalancer run fails due to NPE. Contributed (brahma: rev 6efb8c9c6018ec688eeba8c61c220009ec350a44) (edit) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/balancer/TestBalancer.java
          Hide
          linyiqun Yiqun Lin added a comment -

          Brahma Reddy Battula,thanks for the commit!

          Show
          linyiqun Yiqun Lin added a comment - Brahma Reddy Battula ,thanks for the commit!

            People

            • Assignee:
              linyiqun Yiqun Lin
              Reporter:
              linyiqun Yiqun Lin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development