Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6396

TestPipeApplication fails by NullPointerException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: test
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      TestPipeApplication#testApplication and TestPipeApplication#testRunner fail on trunk.

      Running org.apache.hadoop.mapred.pipes.TestPipeApplication
      Tests run: 5, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 5.758 sec <<< FAILURE! - in org.apache.hadoop.mapred.pipes.TestPipeApplication
      testApplication(org.apache.hadoop.mapred.pipes.TestPipeApplication)  Time elapsed: 1.157 sec  <<< ERROR!
      java.lang.NullPointerException: null
      	at org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
      	at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
      	at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
      	at org.apache.hadoop.mapred.pipes.TestPipeApplication.testApplication(TestPipeApplication.java:180)
      
      testRunner(org.apache.hadoop.mapred.pipes.TestPipeApplication)  Time elapsed: 0.035 sec  <<< ERROR!
      java.lang.NullPointerException: null
      	at org.apache.hadoop.fs.permission.FsPermission.applyUMask(FsPermission.java:213)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:498)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:553)
      	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:526)
      	at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:298)
      	at org.apache.hadoop.fs.RawLocalFileSystem.create(RawLocalFileSystem.java:287)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:891)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:788)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:777)
      	at org.apache.hadoop.mapred.pipes.TestPipeApplication.testRunner(TestPipeApplication.java:98)
      
      1. MAPREDUCE-6396.patch
        2 kB
        Brahma Reddy Battula

        Issue Links

          Activity

          Hide
          ajisakaa Akira Ajisaka added a comment -

          I found this issue was broken by the behavior change of RawLocalFileSystem#mkdir introduced by HADOOP-11347. We should fix the test case.

          Show
          ajisakaa Akira Ajisaka added a comment - I found this issue was broken by the behavior change of RawLocalFileSystem#mkdir introduced by HADOOP-11347 . We should fix the test case.
          Hide
          zxu zhihai xu added a comment -

          It looks like RawLocalFileSystem#umask is not initialized in the test, which cause the NPE. Maybe we should call fs.initialize(FsConstants.LOCAL_FS_URI, conf); instead of fs.setConf(conf); so umask will be initialized.

          Show
          zxu zhihai xu added a comment - It looks like RawLocalFileSystem#umask is not initialized in the test, which cause the NPE. Maybe we should call fs.initialize(FsConstants.LOCAL_FS_URI, conf); instead of fs.setConf(conf); so umask will be initialized.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          thanks zhihai xu for taking a look into this issue and thanks Akira Ajisaka for reporting..Attached the patch ..Kindly review..

          Show
          brahmareddy Brahma Reddy Battula added a comment - thanks zhihai xu for taking a look into this issue and thanks Akira Ajisaka for reporting..Attached the patch ..Kindly review..
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 6m 18s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 tests included 0m 0s The patch appears to include 1 new or modified test files.
          +1 javac 7m 29s There were no new javac warning messages.
          +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings.
          +1 checkstyle 0m 36s There were no new checkstyle issues.
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 32s mvn install still works.
          +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
          +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          -1 mapreduce tests 107m 48s Tests failed in hadoop-mapreduce-client-jobclient.
              125m 26s  



          Reason Tests
          Failed unit tests hadoop.mapred.TestReduceFetch
            hadoop.mapred.TestMerge
            hadoop.mapred.TestMRIntermediateDataEncryption
            hadoop.mapred.TestReduceFetchFromPartialMem



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12739470/MAPREDUCE-6396.patch
          Optional Tests javac unit findbugs checkstyle
          git revision trunk / b8341f1
          hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt
          Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/testReport/
          Java 1.7.0_55
          uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 6m 18s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. +1 tests included 0m 0s The patch appears to include 1 new or modified test files. +1 javac 7m 29s There were no new javac warning messages. +1 release audit 0m 19s The applied patch does not increase the total number of release audit warnings. +1 checkstyle 0m 36s There were no new checkstyle issues. +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 32s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 findbugs 0m 48s The patch does not introduce any new Findbugs (version 3.0.0) warnings. -1 mapreduce tests 107m 48s Tests failed in hadoop-mapreduce-client-jobclient.     125m 26s   Reason Tests Failed unit tests hadoop.mapred.TestReduceFetch   hadoop.mapred.TestMerge   hadoop.mapred.TestMRIntermediateDataEncryption   hadoop.mapred.TestReduceFetchFromPartialMem Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12739470/MAPREDUCE-6396.patch Optional Tests javac unit findbugs checkstyle git revision trunk / b8341f1 hadoop-mapreduce-client-jobclient test log https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/artifact/patchprocess/testrun_hadoop-mapreduce-client-jobclient.txt Test Results https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/testReport/ Java 1.7.0_55 uname Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5803/console This message was automatically generated.
          Hide
          brahmareddy Brahma Reddy Battula added a comment -

          Testcase failures are unrelated..

          Show
          brahmareddy Brahma Reddy Battula added a comment - Testcase failures are unrelated..
          Hide
          zxu zhihai xu added a comment -

          thanks for working on this issue, Brahma Reddy Battula.
          the patch looks good to me, +1 (non-binding)

          Show
          zxu zhihai xu added a comment - thanks for working on this issue, Brahma Reddy Battula . the patch looks good to me, +1 (non-binding)
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Brahma Reddy Battula for creating the patch and thanks zhihai xu for the comment. +1, committing this.

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Brahma Reddy Battula for creating the patch and thanks zhihai xu for the comment. +1, committing this.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk. Thanks all who contributed to this issue.

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk. Thanks all who contributed to this issue.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8019 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8019/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8019 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8019/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #960 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/960/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #960 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/960/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/230/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #230 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/230/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2158/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2158 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2158/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #219 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/219/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #219 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/219/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java hadoop-mapreduce-project/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/228/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/CHANGES.txt
          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #228 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/228/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/CHANGES.txt hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2176 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2176/)
          MAPREDUCE-6396. TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32)

          • hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java
          • hadoop-mapreduce-project/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2176 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2176/ ) MAPREDUCE-6396 . TestPipeApplication fails by NullPointerException. Contributed by Brahma Reddy Battula. (aajisaka: rev 85cc644f92e0e55293993c43e381f161bca5ce32) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/pipes/TestPipeApplication.java hadoop-mapreduce-project/CHANGES.txt

            People

            • Assignee:
              brahmareddy Brahma Reddy Battula
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development