Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12831

LocalFS/FSOutputSummer NPEs in constructor if bytes per checksum set to 0

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.8.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      If you set the number of bytes per checksum to zero,

          conf.setInt(LocalFileSystemConfigKeys.LOCAL_FS_BYTES_PER_CHECKSUM_KEY, 0)
      

      then create a "file://" instance, you get to see a stack trace

      1. HADOOP-12831.000.patch
        3 kB
        Mingliang Liu
      2. HADOOP-12831.001.patch
        2 kB
        Mingliang Liu
      3. HADOOP-12831.002.patch
        3 kB
        Mingliang Liu

        Activity

        Hide
        stevel@apache.org Steve Loughran added a comment -

        I had hoped this would be a way to disable checksumming and buffering on file:, but it instead I found a new way to break things

         contains 1 event(s)
        java.lang.NullPointerException
        	at org.apache.hadoop.fs.FSOutputSummer.<init>(FSOutputSummer.java:54)
        	at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:390)
        	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456)
        	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435)
        	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:917)
        	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:898)
        	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:795)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.createLogFileStream(FileSystemTimelineWriter.java:406)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.prepareForWrite(FileSystemTimelineWriter.java:386)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.<init>(FileSystemTimelineWriter.java:363)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$EntityLogFD.<init>(FileSystemTimelineWriter.java:329)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.createSummaryFDAndWrite(FileSystemTimelineWriter.java:842)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummmaryEntityLogs(FileSystemTimelineWriter.java:826)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummaryEntityLogs(FileSystemTimelineWriter.java:805)
        	at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter.putEntities(FileSystemTimelineWriter.java:222)
        	at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:658)
        	at org.apache.spark.deploy.history.yarn.YarnHistoryService.postOneEntity(YarnHistoryService.scala:825)
        	at org.apache.spark.deploy.history.yarn.YarnHistoryService.org$apache$spark$deploy$history$yarn$YarnHistoryService$$postEntities(YarnHistoryService.scala:899)
        	at org.apache.spark.deploy.history.yarn.YarnHistoryService$EntityPoster.run(YarnHistoryService.scala:1105)
        
        Show
        stevel@apache.org Steve Loughran added a comment - I had hoped this would be a way to disable checksumming and buffering on file: , but it instead I found a new way to break things contains 1 event(s) java.lang.NullPointerException at org.apache.hadoop.fs.FSOutputSummer.<init>(FSOutputSummer.java:54) at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSOutputSummer.<init>(ChecksumFileSystem.java:390) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:456) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:435) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:917) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:898) at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:795) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.createLogFileStream(FileSystemTimelineWriter.java:406) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.prepareForWrite(FileSystemTimelineWriter.java:386) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFD.<init>(FileSystemTimelineWriter.java:363) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$EntityLogFD.<init>(FileSystemTimelineWriter.java:329) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.createSummaryFDAndWrite(FileSystemTimelineWriter.java:842) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummmaryEntityLogs(FileSystemTimelineWriter.java:826) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter$LogFDsCache.writeSummaryEntityLogs(FileSystemTimelineWriter.java:805) at org.apache.hadoop.yarn.client.api.impl.FileSystemTimelineWriter.putEntities(FileSystemTimelineWriter.java:222) at org.apache.hadoop.yarn.client.api.impl.TimelineClientImpl.putEntities(TimelineClientImpl.java:658) at org.apache.spark.deploy.history.yarn.YarnHistoryService.postOneEntity(YarnHistoryService.scala:825) at org.apache.spark.deploy.history.yarn.YarnHistoryService.org$apache$spark$deploy$history$yarn$YarnHistoryService$$postEntities(YarnHistoryService.scala:899) at org.apache.spark.deploy.history.yarn.YarnHistoryService$EntityPoster.run(YarnHistoryService.scala:1105)
        Hide
        liuml07 Mingliang Liu added a comment -

        It looks like in DataChecksum#newDataChecksum, returning null is by desgin:

          public static DataChecksum newDataChecksum(Type type, int bytesPerChecksum ) {
            if ( bytesPerChecksum <= 0 ) {
              return null;
            }
        ...
        

        The reason may be that, the bytesPerChecksum was used as divisor for calculating number of chunks and checksum buffer size. We can document this, or throw exception with a user-friendly message for this.

        I think this is not feasible to disable checksumming and buffering in this way. Will RawLocalFileSystem work in your case?

        Show
        liuml07 Mingliang Liu added a comment - It looks like in DataChecksum#newDataChecksum , returning null is by desgin: public static DataChecksum newDataChecksum(Type type, int bytesPerChecksum ) { if ( bytesPerChecksum <= 0 ) { return null ; } ... The reason may be that, the bytesPerChecksum was used as divisor for calculating number of chunks and checksum buffer size. We can document this, or throw exception with a user-friendly message for this. I think this is not feasible to disable checksumming and buffering in this way. Will RawLocalFileSystem work in your case?
        Hide
        stevel@apache.org Steve Loughran added a comment -

        oh, it NPEs early on. Its' not fixable...I'm just looking for a way to force raw local/bypass checksumming and its buffering on a bit of ATS that I can't get at directly: it's not my code creating the FS. It also means I can't get at the rawlocalfs. I'll have to think of another way.

        regarding this issue, a precondition saying "checksum block size too small" is the only thing that makes sense. It will fail fast but with a meaningful message.

        Show
        stevel@apache.org Steve Loughran added a comment - oh, it NPEs early on. Its' not fixable...I'm just looking for a way to force raw local/bypass checksumming and its buffering on a bit of ATS that I can't get at directly: it's not my code creating the FS. It also means I can't get at the rawlocalfs. I'll have to think of another way. regarding this issue, a precondition saying "checksum block size too small" is the only thing that makes sense. It will fail fast but with a meaningful message.
        Hide
        liuml07 Mingliang Liu added a comment -

        How about remapping the implementation for file URIs globally by setting the property fs.file.impl to the value org.apache.hadoop.fs.RawLocalFileSystem? It may be so heavy that it is useful in few cases. p.s. I've never tried this before.

        Show
        liuml07 Mingliang Liu added a comment - How about remapping the implementation for file URIs globally by setting the property fs.file.impl to the value org.apache.hadoop.fs.RawLocalFileSystem ? It may be so heavy that it is useful in few cases. p.s. I've never tried this before.
        Hide
        gtCarrera9 Li Lu added a comment -

        Thanks for the quick patch Mingliang Liu! Yes a precondition may make sense.

        One nit:

        bytes per checksum too small
        

        Maybe we can explicitly tell the user what is the valid range of the value? "Too small" looks very vague.

        Show
        gtCarrera9 Li Lu added a comment - Thanks for the quick patch Mingliang Liu ! Yes a precondition may make sense. One nit: bytes per checksum too small Maybe we can explicitly tell the user what is the valid range of the value? "Too small" looks very vague.
        Hide
        liuml07 Mingliang Liu added a comment -

        Thanks Li Lu for the prompt review. The v1 patch is to address the comments.

        Show
        liuml07 Mingliang Liu added a comment - Thanks Li Lu for the prompt review. The v1 patch is to address the comments.
        Hide
        gtCarrera9 Li Lu added a comment -

        Thanks. Sorry I missed one point just now, DataChecksum appears to be a little bit too general for this fix?

        Also, Steve Loughran would you mind to double check it to see if this is the proper fix? Thanks!

        Show
        gtCarrera9 Li Lu added a comment - Thanks. Sorry I missed one point just now, DataChecksum appears to be a little bit too general for this fix? Also, Steve Loughran would you mind to double check it to see if this is the proper fix? Thanks!
        Hide
        liuml07 Mingliang Liu added a comment -

        Thanks Li Lu for proposing the concern.

        In the current patch, my intension was that there were a few other callsites that create a DataChecksum object but do not check the null return value. Users may meet similar problems with vague NPE message in these places.

        It makes sense to me to narrow the change to the ChecksumFileSystem as I'm not 100% sure about the DataChecksum#newDataChecksum precondition.

        Show
        liuml07 Mingliang Liu added a comment - Thanks Li Lu for proposing the concern. In the current patch, my intension was that there were a few other callsites that create a DataChecksum object but do not check the null return value. Users may meet similar problems with vague NPE message in these places. It makes sense to me to narrow the change to the ChecksumFileSystem as I'm not 100% sure about the DataChecksum#newDataChecksum precondition.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 12m 26s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 6m 32s trunk passed
        +1 compile 5m 46s trunk passed with JDK v1.8.0_72
        +1 compile 6m 37s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 1m 2s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 33s trunk passed
        +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72
        +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 39s the patch passed
        +1 compile 5m 54s the patch passed with JDK v1.8.0_72
        +1 javac 5m 54s the patch passed
        +1 compile 6m 43s the patch passed with JDK v1.7.0_95
        +1 javac 6m 43s the patch passed
        +1 checkstyle 0m 21s hadoop-common-project/hadoop-common: patch generated 0 new + 131 unchanged - 2 fixed = 131 total (was 133)
        +1 mvnsite 1m 1s the patch passed
        +1 mvneclipse 0m 13s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 48s the patch passed
        +1 javadoc 0m 51s the patch passed with JDK v1.8.0_72
        +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95
        -1 unit 16m 24s hadoop-common in the patch failed with JDK v1.8.0_72.
        +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95.
        +1 asflicense 0m 20s Patch does not generate ASF License warnings.
        80m 9s



        Reason Tests
        JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12789321/HADOOP-12831.001.patch
        JIRA Issue HADOOP-12831
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux a95313e65706 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / def754e
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 26s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 6m 32s trunk passed +1 compile 5m 46s trunk passed with JDK v1.8.0_72 +1 compile 6m 37s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 2s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 33s trunk passed +1 javadoc 0m 51s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 5s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 39s the patch passed +1 compile 5m 54s the patch passed with JDK v1.8.0_72 +1 javac 5m 54s the patch passed +1 compile 6m 43s the patch passed with JDK v1.7.0_95 +1 javac 6m 43s the patch passed +1 checkstyle 0m 21s hadoop-common-project/hadoop-common: patch generated 0 new + 131 unchanged - 2 fixed = 131 total (was 133) +1 mvnsite 1m 1s the patch passed +1 mvneclipse 0m 13s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 48s the patch passed +1 javadoc 0m 51s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 3s the patch passed with JDK v1.7.0_95 -1 unit 16m 24s hadoop-common in the patch failed with JDK v1.8.0_72. +1 unit 7m 16s hadoop-common in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 80m 9s Reason Tests JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12789321/HADOOP-12831.001.patch JIRA Issue HADOOP-12831 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux a95313e65706 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / def754e Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8697/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment -

        The v2 patch is to check the config key in ChecksumFileSystem to narrow down the change, as Li Lu suggested.

        Show
        liuml07 Mingliang Liu added a comment - The v2 patch is to check the config key in ChecksumFileSystem to narrow down the change, as Li Lu suggested.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 13s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        +1 mvninstall 8m 2s trunk passed
        +1 compile 9m 29s trunk passed with JDK v1.8.0_72
        +1 compile 8m 12s trunk passed with JDK v1.7.0_95
        +1 checkstyle 0m 22s trunk passed
        +1 mvnsite 1m 11s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 findbugs 1m 42s trunk passed
        +1 javadoc 1m 0s trunk passed with JDK v1.8.0_72
        +1 javadoc 1m 8s trunk passed with JDK v1.7.0_95
        +1 mvninstall 0m 45s the patch passed
        +1 compile 6m 47s the patch passed with JDK v1.8.0_72
        +1 javac 6m 47s the patch passed
        +1 compile 7m 25s the patch passed with JDK v1.7.0_95
        +1 javac 7m 25s the patch passed
        +1 checkstyle 0m 23s the patch passed
        +1 mvnsite 1m 6s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 findbugs 1m 57s the patch passed
        +1 javadoc 0m 57s the patch passed with JDK v1.8.0_72
        +1 javadoc 1m 9s the patch passed with JDK v1.7.0_95
        +1 unit 9m 6s hadoop-common in the patch passed with JDK v1.8.0_72.
        -1 unit 8m 10s hadoop-common in the patch failed with JDK v1.7.0_95.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        71m 8s



        Reason Tests
        JDK v1.7.0_95 Failed junit tests hadoop.fs.TestSymlinkLocalFSFileSystem



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790205/HADOOP-12831.002.patch
        JIRA Issue HADOOP-12831
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux dd5e668108a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6b0f813
        Default Java 1.7.0_95
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt
        JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 8m 2s trunk passed +1 compile 9m 29s trunk passed with JDK v1.8.0_72 +1 compile 8m 12s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 22s trunk passed +1 mvnsite 1m 11s trunk passed +1 mvneclipse 0m 14s trunk passed +1 findbugs 1m 42s trunk passed +1 javadoc 1m 0s trunk passed with JDK v1.8.0_72 +1 javadoc 1m 8s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 45s the patch passed +1 compile 6m 47s the patch passed with JDK v1.8.0_72 +1 javac 6m 47s the patch passed +1 compile 7m 25s the patch passed with JDK v1.7.0_95 +1 javac 7m 25s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 6s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 1m 57s the patch passed +1 javadoc 0m 57s the patch passed with JDK v1.8.0_72 +1 javadoc 1m 9s the patch passed with JDK v1.7.0_95 +1 unit 9m 6s hadoop-common in the patch passed with JDK v1.8.0_72. -1 unit 8m 10s hadoop-common in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 71m 8s Reason Tests JDK v1.7.0_95 Failed junit tests hadoop.fs.TestSymlinkLocalFSFileSystem Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790205/HADOOP-12831.002.patch JIRA Issue HADOOP-12831 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux dd5e668108a2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6b0f813 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8732/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        liuml07 Mingliang Liu added a comment -

        Test failure is not related. Specially, I filed HADOOP-12849 to track the failing test.

        Show
        liuml07 Mingliang Liu added a comment - Test failure is not related. Specially, I filed HADOOP-12849 to track the failing test.
        Hide
        gtCarrera9 Li Lu added a comment -

        Latest patch LGTM. Will wait for about a day then commit. Steve Loughran I believe you were talking about a fix like this, so would you please double check it? Thanks!

        Show
        gtCarrera9 Li Lu added a comment - Latest patch LGTM. Will wait for about a day then commit. Steve Loughran I believe you were talking about a fix like this, so would you please double check it? Thanks!
        Hide
        stevel@apache.org Steve Loughran added a comment -

        +1

        Show
        stevel@apache.org Steve Loughran added a comment - +1
        Hide
        gtCarrera9 Li Lu added a comment -

        +1, will commit shortly.

        Show
        gtCarrera9 Li Lu added a comment - +1, will commit shortly.
        Hide
        gtCarrera9 Li Lu added a comment -

        I have committed the latest patch into trunk, branch-2 and branch-2.8. Thanks Mingliang Liu for the contribution. Thanks Steve Loughran for the review!

        Show
        gtCarrera9 Li Lu added a comment - I have committed the latest patch into trunk, branch-2 and branch-2.8. Thanks Mingliang Liu for the contribution. Thanks Steve Loughran for the review!
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9383 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9383/)
        HADOOP-12831. LocalFS/FSOutputSummer NPEs in constructor if bytes per (gtcarrera9: rev 7545ce6636066a05763744a817878e03ee87f456)

        • hadoop-common-project/hadoop-common/CHANGES.txt
        • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java
        • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestChecksumFileSystem.java
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9383 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9383/ ) HADOOP-12831 . LocalFS/FSOutputSummer NPEs in constructor if bytes per (gtcarrera9: rev 7545ce6636066a05763744a817878e03ee87f456) hadoop-common-project/hadoop-common/CHANGES.txt hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestChecksumFileSystem.java
        Hide
        liuml07 Mingliang Liu added a comment -

        Thank you Li Lu for reviewing and committing the patch. Thank you Steve Loughran for reporting and reviewing.

        Show
        liuml07 Mingliang Liu added a comment - Thank you Li Lu for reviewing and committing the patch. Thank you Steve Loughran for reporting and reviewing.

          People

          • Assignee:
            liuml07 Mingliang Liu
            Reporter:
            stevel@apache.org Steve Loughran
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development