Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Environment:

      jenkins

    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      NPE surfacing in MiniDFSCluster.shutdown during test teardown

      1. HDFS-8043-003.patch
        1 kB
        Brahma Reddy Battula
      2. HDFS-8043-002.patch
        1 kB
        Brahma Reddy Battula
      3. HDFS-8043.patch
        0.7 kB
        Brahma Reddy Battula

        Activity

        Hide
        stevel@apache.org Steve Loughran added a comment -

        Stack from https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk/lastCompletedBuild/testReport/org.apache.hadoop.hdfs/TestDFSClientFailover/testDfsClientFailover/

        java.lang.NullPointerException: null
        	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1750)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1711)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1704)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:842)
        	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:471)
        	at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:430)
        	at org.apache.hadoop.hdfs.TestDFSClientFailover.setUpCluster(TestDFSClientFailover.java:81)
        
        Show
        stevel@apache.org Steve Loughran added a comment - Stack from https://builds.apache.org/view/H-L/view/Hadoop/job/Hadoop-Hdfs-trunk/lastCompletedBuild/testReport/org.apache.hadoop.hdfs/TestDFSClientFailover/testDfsClientFailover/ java.lang.NullPointerException: null at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1750) at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1711) at org.apache.hadoop.hdfs.MiniDFSCluster.shutdown(MiniDFSCluster.java:1704) at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:842) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:471) at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:430) at org.apache.hadoop.hdfs.TestDFSClientFailover.setUpCluster(TestDFSClientFailover.java:81)
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Steve Loughran Thanks a lot for reporting this jira..Attached the patch, kindly review the same... Thanks.

        Show
        brahmareddy Brahma Reddy Battula added a comment - Steve Loughran Thanks a lot for reporting this jira..Attached the patch, kindly review the same... Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12709270/HDFS-8043.patch
        against trunk revision dc5596c.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 1 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10173//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10173//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12709270/HDFS-8043.patch against trunk revision dc5596c. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10173//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10173//console This message is automatically generated.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Brahma Reddy Battula thank you for taking this issue. I think we should fix MiniDFSCluster itself since NullPointerException occurs in MiniDFSCluster class instead of test class. IIUC, base_dir in MiniDFSCluster is null but base_dir.delete() or base_dir.deleteOnExit() is called. Could you check it?

        Show
        ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula thank you for taking this issue. I think we should fix MiniDFSCluster itself since NullPointerException occurs in MiniDFSCluster class instead of test class. IIUC, base_dir in MiniDFSCluster is null but base_dir.delete() or base_dir.deleteOnExit() is called. Could you check it?
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Tsuyoshi Ozawa Thanks for taking a look into this issue.. Yes , you are correct..Now I covered both...Kindly review..

        Show
        brahmareddy Brahma Reddy Battula added a comment - Tsuyoshi Ozawa Thanks for taking a look into this issue.. Yes , you are correct..Now I covered both...Kindly review..
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Brahma Reddy Battula thanks for your updating.

        -    if (deleteDfsDir) {
        +    if (deleteDfsDir && base_dir != null) {
                 base_dir.delete();
             } else {
        +      if (base_dir != null) {
                 base_dir.deleteOnExit();
        +      }
             }
        

        The condition statements look complex. Could you fix it not to use nested if-else statement?

        Show
        ozawa Tsuyoshi Ozawa added a comment - Brahma Reddy Battula thanks for your updating. - if (deleteDfsDir) { + if (deleteDfsDir && base_dir != null ) { base_dir.delete(); } else { + if (base_dir != null ) { base_dir.deleteOnExit(); + } } The condition statements look complex. Could you fix it not to use nested if-else statement?
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726184/HDFS-8043-002.patch
        against trunk revision 76e7264.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10301//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10301//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726184/HDFS-8043-002.patch against trunk revision 76e7264. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10301//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10301//console This message is automatically generated.
        Hide
        stevel@apache.org Steve Loughran added a comment -

        you can reorder the condition to say

        
        

        if (basedir != null) {
        if (deleteDfsDir)

        { delete-now }

        else

        {delete-later}


        }

        It'll be less confusing that way

        Show
        stevel@apache.org Steve Loughran added a comment - you can reorder the condition to say if (basedir != null) { if (deleteDfsDir) { delete-now } else {delete-later} } It'll be less confusing that way
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Thanks a lot Tsuyoshi Ozawa and Steve Loughran for reviews.. Updated the patch..Kindly review...

        Show
        brahmareddy Brahma Reddy Battula added a comment - Thanks a lot Tsuyoshi Ozawa and Steve Loughran for reviews.. Updated the patch..Kindly review...
        Hide
        hadoopqa Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12726378/HDFS-8043-003.patch
        against trunk revision f47a576.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10312//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10312//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726378/HDFS-8043-003.patch against trunk revision f47a576. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 2 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-hdfs-project/hadoop-hdfs. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10312//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10312//console This message is automatically generated.
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        +1

        Show
        ozawa Tsuyoshi Ozawa added a comment - +1
        Hide
        ozawa Tsuyoshi Ozawa added a comment -

        Committed this to trunk and branch-2. Thanks Brahma Reddy Battula for your contribution and thanks Steve Loughran for your review.

        Show
        ozawa Tsuyoshi Ozawa added a comment - Committed this to trunk and branch-2. Thanks Brahma Reddy Battula for your contribution and thanks Steve Loughran for your review.
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #7613 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7613/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #7613 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7613/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/168/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #168 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/168/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Yarn-trunk #902 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/902/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #902 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/902/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk #2100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2100/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2100 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2100/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/159/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #159 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/159/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #169 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/169/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #169 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/169/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2118 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2118/)
        HDFS-8043. NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4)

        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java
        • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java
        • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2118 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2118/ ) HDFS-8043 . NPE in MiniDFSCluster teardown. Contributed by Brahma Reddy Battula. (ozawa: rev 5459b241c86cc9a26fecca9a06ceaf524e48fed4) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDFSClientFailover.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/MiniDFSCluster.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt

          People

          • Assignee:
            brahmareddy Brahma Reddy Battula
            Reporter:
            stevel@apache.org Steve Loughran
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development