Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11152

Start erasure coding policy ID number from 1 instead of 0 to void potential unexpected errors

    Details

    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      The NameNode metadata for storing erasure coding policies has changed.

      Description

      This task will change erasure coding policy ID number starting from 1 instead of current 0, to avoid some potential unexpected errors in codes since 0 is default value for integer variables.

      1. HDFS-11152-v1.patch
        1 kB
        SammiChen
      2. HDFS-11152-v2.patch
        4 kB
        SammiChen
      3. HDFS-11152-v3.patch
        2 kB
        SammiChen
      4. HDFS-11152-v4.patch
        2 kB
        SammiChen

        Activity

        Hide
        Sammi SammiChen added a comment -

        Initial patch

        Show
        Sammi SammiChen added a comment - Initial patch
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 mvninstall 7m 23s trunk passed
        +1 compile 0m 29s trunk passed
        +1 checkstyle 0m 15s trunk passed
        +1 mvnsite 0m 32s trunk passed
        +1 mvneclipse 0m 13s trunk passed
        +1 findbugs 1m 24s trunk passed
        +1 javadoc 0m 20s trunk passed
        +1 mvninstall 0m 29s the patch passed
        +1 compile 0m 28s the patch passed
        +1 javac 0m 28s the patch passed
        +1 checkstyle 0m 11s the patch passed
        +1 mvnsite 0m 31s the patch passed
        +1 mvneclipse 0m 9s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 1m 29s the patch passed
        +1 javadoc 0m 16s the patch passed
        +1 unit 0m 52s hadoop-hdfs-client in the patch passed.
        +1 asflicense 0m 16s The patch does not generate ASF License warnings.
        16m 38s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11152
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839956/HDFS-11152-v1.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 5ad978252f29 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 6f80742
        Default Java 1.8.0_111
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17629/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17629/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 mvninstall 7m 23s trunk passed +1 compile 0m 29s trunk passed +1 checkstyle 0m 15s trunk passed +1 mvnsite 0m 32s trunk passed +1 mvneclipse 0m 13s trunk passed +1 findbugs 1m 24s trunk passed +1 javadoc 0m 20s trunk passed +1 mvninstall 0m 29s the patch passed +1 compile 0m 28s the patch passed +1 javac 0m 28s the patch passed +1 checkstyle 0m 11s the patch passed +1 mvnsite 0m 31s the patch passed +1 mvneclipse 0m 9s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 1m 29s the patch passed +1 javadoc 0m 16s the patch passed +1 unit 0m 52s hadoop-hdfs-client in the patch passed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 16m 38s Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11152 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12839956/HDFS-11152-v1.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 5ad978252f29 3.13.0-93-generic #140-Ubuntu SMP Mon Jul 18 21:21:05 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 6f80742 Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17629/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client U: hadoop-hdfs-project/hadoop-hdfs-client Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17629/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Hi Sammi, thanks for the patch. Looks good overall, but are there places we should add Precondition checks (e.g. fsimage/editlog loading, file read and write) for correctness? Unit tests for these situations would also be good.

        Show
        andrew.wang Andrew Wang added a comment - Hi Sammi, thanks for the patch. Looks good overall, but are there places we should add Precondition checks (e.g. fsimage/editlog loading, file read and write) for correctness? Unit tests for these situations would also be good.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Ping. Yeah, I agree some extra regression tests are greatly appreciated.

        Show
        jojochuang Wei-Chiu Chuang added a comment - Ping. Yeah, I agree some extra regression tests are greatly appreciated.
        Hide
        Sammi SammiChen added a comment -

        Hi Andrew and Wei-Chiu, thanks for your advice. Sure, I will upload a new patch, adding test cases.

        Show
        Sammi SammiChen added a comment - Hi Andrew and Wei-Chiu, thanks for your advice. Sure, I will upload a new patch, adding test cases.
        Hide
        Sammi SammiChen added a comment -

        I just find one FSImage issue regarding erasure coding. I created JIRA HDFS-11268. I think I should solve HDFS-11268 first, then come back to this JIRA.

        Show
        Sammi SammiChen added a comment - I just find one FSImage issue regarding erasure coding. I created JIRA HDFS-11268 . I think I should solve HDFS-11268 first, then come back to this JIRA.
        Hide
        Sammi SammiChen added a comment -

        Hi Andrew Wang] I did some investigation. It seems EC policy ID is not widely used. For examples,
        1. For directory, EC policy is saved as XAttr using policy name(policy name doesn't include policy ID). When directory is saved to FsImage, all its XAttrs are saved including EC policy name.
        2. For file, EC policy ID is saved in head. When file is saved to FsImage, EC policy ID is saved in replication field.
        3. The setErasureCodingPolicy API, user will provide a ErasureCodingPolicy parameter, currently only the schema and cell size is used to check if it's a legitimate EC policy, policy ID is not involved in parameter checking. Because currently setErasureCodingPolicy can only applies to directory. And only policy name is saved on directory. I will enforce the policy ID check in HDFS-10996 implementation which specify per-file policy when create file.
        4. File read/write. Double checked DFSStripedInputStream and DFSStripedOutputStream, policy ID is not used at all. The schema and cellsize are used.

        So, later I plan to upload a new patch, adding Precondition checks on FsImage file loading only. Because other spaces use policy name, schema and cell size.

        Show
        Sammi SammiChen added a comment - Hi Andrew Wang ] I did some investigation. It seems EC policy ID is not widely used. For examples, 1. For directory, EC policy is saved as XAttr using policy name(policy name doesn't include policy ID). When directory is saved to FsImage, all its XAttrs are saved including EC policy name. 2. For file, EC policy ID is saved in head. When file is saved to FsImage, EC policy ID is saved in replication field. 3. The setErasureCodingPolicy API, user will provide a ErasureCodingPolicy parameter, currently only the schema and cell size is used to check if it's a legitimate EC policy, policy ID is not involved in parameter checking. Because currently setErasureCodingPolicy can only applies to directory. And only policy name is saved on directory. I will enforce the policy ID check in HDFS-10996 implementation which specify per-file policy when create file. 4. File read/write. Double checked DFSStripedInputStream and DFSStripedOutputStream , policy ID is not used at all. The schema and cellsize are used. So, later I plan to upload a new patch, adding Precondition checks on FsImage file loading only. Because other spaces use policy name, schema and cell size.
        Hide
        Sammi SammiChen added a comment -

        1. Add EC policy ID check when loading file from fsImage
        2. Add EC policy ID check when set policy on path

        Show
        Sammi SammiChen added a comment - 1. Add EC policy ID check when loading file from fsImage 2. Add EC policy ID check when set policy on path
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 34s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
        0 mvndep 0m 10s Maven dependency ordering for branch
        +1 mvninstall 13m 15s trunk passed
        +1 compile 1m 24s trunk passed
        +1 checkstyle 0m 30s trunk passed
        +1 mvnsite 1m 24s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 6s trunk passed
        +1 javadoc 0m 57s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 14s the patch passed
        +1 compile 1m 18s the patch passed
        +1 javac 1m 18s the patch passed
        +1 checkstyle 0m 27s the patch passed
        +1 mvnsite 1m 17s the patch passed
        +1 mvneclipse 0m 20s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 15s the patch passed
        +1 javadoc 0m 54s the patch passed
        +1 unit 0m 53s hadoop-hdfs-client in the patch passed.
        -1 unit 102m 36s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        135m 50s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140
          hadoop.hdfs.server.datanode.TestFsDatasetCache
          hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050
        Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2
          org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11152
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848501/HDFS-11152-v2.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux f8f62651163e 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / e015b56
        Default Java 1.8.0_111
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18224/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18224/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18224/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 34s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 13m 15s trunk passed +1 compile 1m 24s trunk passed +1 checkstyle 0m 30s trunk passed +1 mvnsite 1m 24s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 6s trunk passed +1 javadoc 0m 57s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 14s the patch passed +1 compile 1m 18s the patch passed +1 javac 1m 18s the patch passed +1 checkstyle 0m 27s the patch passed +1 mvnsite 1m 17s the patch passed +1 mvneclipse 0m 20s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 15s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 0m 53s hadoop-hdfs-client in the patch passed. -1 unit 102m 36s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 135m 50s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure140   hadoop.hdfs.server.datanode.TestFsDatasetCache   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure050 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2   org.apache.hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11152 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12848501/HDFS-11152-v2.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux f8f62651163e 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / e015b56 Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18224/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18224/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18224/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        Marking ec must do's as blockers for alpha3

        Show
        andrew.wang Andrew Wang added a comment - Marking ec must do's as blockers for alpha3
        Hide
        jojochuang Wei-Chiu Chuang added a comment - - edited

        I was out for an extended period of time, and have been catching up since then.
        I'll review this patch soon. Thanks!

        Show
        jojochuang Wei-Chiu Chuang added a comment - - edited I was out for an extended period of time, and have been catching up since then. I'll review this patch soon. Thanks!
        Hide
        Sammi SammiChen added a comment - - edited

        Hi Wei-Chiu Chuang, thanks for review the patch! A new patch is available. I gone through all EC policy related modules. It seems when user sets a EC policy, only the policy name matters. The policy ID is just used internally to save memory space. So an invalid policy only happens when the fsImage file is tempered or editlog is tempered. Is there any known test utility routine which can temper a fsimage or editlog, change a specific filed?

        Show
        Sammi SammiChen added a comment - - edited Hi Wei-Chiu Chuang , thanks for review the patch! A new patch is available. I gone through all EC policy related modules. It seems when user sets a EC policy, only the policy name matters. The policy ID is just used internally to save memory space. So an invalid policy only happens when the fsImage file is tempered or editlog is tempered. Is there any known test utility routine which can temper a fsimage or editlog, change a specific filed?
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        We've made quite some big changes in the past few weeks.
        It is certainly possible to use OfflineImageViewer to create a fsimage.

        Show
        jojochuang Wei-Chiu Chuang added a comment - We've made quite some big changes in the past few weeks. It is certainly possible to use OfflineImageViewer to create a fsimage.
        Hide
        andrew.wang Andrew Wang added a comment -

        There is a possibility for invalid IDs if people use an old version of OIV against a newer FSImage. Normally, this kind of change should also bump the NN LayoutVersion to guard against this kind of error, but it doesn't hurt to have additional checks here just in case. Also during the alpha process, we haven't been bumping the LV when making metadata changes since it's still in alpha.

        Show
        andrew.wang Andrew Wang added a comment - There is a possibility for invalid IDs if people use an old version of OIV against a newer FSImage. Normally, this kind of change should also bump the NN LayoutVersion to guard against this kind of error, but it doesn't hurt to have additional checks here just in case. Also during the alpha process, we haven't been bumping the LV when making metadata changes since it's still in alpha.
        Hide
        jojochuang Wei-Chiu Chuang added a comment -

        Retriggered jenkins build https://builds.apache.org/job/PreCommit-HDFS-Build/18628/

        I think the patch is mostly good. We can improve the error message a little bit. This message doesn't tells us much other than the policy id is unexpected. Can we also print the INode id/file name as well as the policy id?

        Show
        jojochuang Wei-Chiu Chuang added a comment - Retriggered jenkins build https://builds.apache.org/job/PreCommit-HDFS-Build/18628/ I think the patch is mostly good. We can improve the error message a little bit. This message doesn't tells us much other than the policy id is unexpected. Can we also print the INode id/file name as well as the policy id?
        Hide
        andrew.wang Andrew Wang added a comment -

        I retriggered one more time, note that you should not put "HDFS-" in the Jenkins parameter field.

        Show
        andrew.wang Andrew Wang added a comment - I retriggered one more time, note that you should not put "HDFS-" in the Jenkins parameter field.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 28s Maven dependency ordering for branch
        +1 mvninstall 14m 33s trunk passed
        +1 compile 1m 39s trunk passed
        +1 checkstyle 0m 41s trunk passed
        +1 mvnsite 1m 37s trunk passed
        +1 mvneclipse 0m 27s trunk passed
        +1 findbugs 3m 38s trunk passed
        +1 javadoc 1m 4s trunk passed
        0 mvndep 0m 7s Maven dependency ordering for patch
        +1 mvninstall 1m 32s the patch passed
        +1 compile 1m 43s the patch passed
        +1 javac 1m 43s the patch passed
        +1 checkstyle 0m 46s the patch passed
        +1 mvnsite 1m 38s the patch passed
        +1 mvneclipse 0m 22s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 4m 7s the patch passed
        +1 javadoc 1m 8s the patch passed
        +1 unit 1m 10s hadoop-hdfs-client in the patch passed.
        -1 unit 85m 10s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 23s The patch does not generate ASF License warnings.
        124m 10s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker
          hadoop.hdfs.server.datanode.TestDirectoryScanner
          hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting
          hadoop.hdfs.server.namenode.ha.TestHAAppend



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11152
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856256/HDFS-11152-v3.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux 437aeb335852 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 28daaf0
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18634/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18634/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18634/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 14m 33s trunk passed +1 compile 1m 39s trunk passed +1 checkstyle 0m 41s trunk passed +1 mvnsite 1m 37s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 38s trunk passed +1 javadoc 1m 4s trunk passed 0 mvndep 0m 7s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 1m 43s the patch passed +1 javac 1m 43s the patch passed +1 checkstyle 0m 46s the patch passed +1 mvnsite 1m 38s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 4m 7s the patch passed +1 javadoc 1m 8s the patch passed +1 unit 1m 10s hadoop-hdfs-client in the patch passed. -1 unit 85m 10s hadoop-hdfs in the patch failed. +1 asflicense 0m 23s The patch does not generate ASF License warnings. 124m 10s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.checker.TestThrottledAsyncChecker   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting   hadoop.hdfs.server.namenode.ha.TestHAAppend Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11152 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856256/HDFS-11152-v3.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 437aeb335852 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 28daaf0 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18634/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18634/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18634/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        Sammi SammiChen added a comment -

        Thanks Wei-Chiu Chuang for the feedback. I uploaded a new patch to refine the error message. Also double checked failed unit tests, they are irrelevant.

        Show
        Sammi SammiChen added a comment - Thanks Wei-Chiu Chuang for the feedback. I uploaded a new patch to refine the error message. Also double checked failed unit tests, they are irrelevant.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 19s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 6s Maven dependency ordering for branch
        +1 mvninstall 12m 26s trunk passed
        +1 compile 1m 21s trunk passed
        +1 checkstyle 0m 39s trunk passed
        +1 mvnsite 1m 23s trunk passed
        +1 mvneclipse 0m 25s trunk passed
        +1 findbugs 3m 7s trunk passed
        +1 javadoc 0m 59s trunk passed
        0 mvndep 0m 6s Maven dependency ordering for patch
        +1 mvninstall 1m 15s the patch passed
        +1 compile 1m 20s the patch passed
        +1 javac 1m 20s the patch passed
        +1 checkstyle 0m 36s the patch passed
        +1 mvnsite 1m 19s the patch passed
        +1 mvneclipse 0m 19s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 findbugs 3m 19s the patch passed
        +1 javadoc 0m 54s the patch passed
        +1 unit 0m 54s hadoop-hdfs-client in the patch passed.
        -1 unit 86m 6s hadoop-hdfs in the patch failed.
        +1 asflicense 0m 18s The patch does not generate ASF License warnings.
        118m 33s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:a9ad5d6
        JIRA Issue HDFS-11152
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856797/HDFS-11152-v4.patch
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
        uname Linux b459c8348057 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 1eb8186
        Default Java 1.8.0_121
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/18641/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18641/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18641/console
        Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 6s Maven dependency ordering for branch +1 mvninstall 12m 26s trunk passed +1 compile 1m 21s trunk passed +1 checkstyle 0m 39s trunk passed +1 mvnsite 1m 23s trunk passed +1 mvneclipse 0m 25s trunk passed +1 findbugs 3m 7s trunk passed +1 javadoc 0m 59s trunk passed 0 mvndep 0m 6s Maven dependency ordering for patch +1 mvninstall 1m 15s the patch passed +1 compile 1m 20s the patch passed +1 javac 1m 20s the patch passed +1 checkstyle 0m 36s the patch passed +1 mvnsite 1m 19s the patch passed +1 mvneclipse 0m 19s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 findbugs 3m 19s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 0m 54s hadoop-hdfs-client in the patch passed. -1 unit 86m 6s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 118m 33s Reason Tests Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeVolumeFailureReporting Subsystem Report/Notes Docker Image:yetus/hadoop:a9ad5d6 JIRA Issue HDFS-11152 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856797/HDFS-11152-v4.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux b459c8348057 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1eb8186 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/18641/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/18641/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/18641/console Powered by Apache Yetus 0.5.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        andrew.wang Andrew Wang added a comment -

        LGTM +1, will commit shortly.

        Show
        andrew.wang Andrew Wang added a comment - LGTM +1, will commit shortly.
        Hide
        andrew.wang Andrew Wang added a comment -

        Committed to trunk, thanks for the contribution Sammi and for reviewing Wei-Chiu!

        Show
        andrew.wang Andrew Wang added a comment - Committed to trunk, thanks for the contribution Sammi and for reviewing Wei-Chiu!
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11371 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11371/)
        HDFS-11152. Start erasure coding policy ID number from 1 instead of 0 to (wang: rev 5addacb1e301991a8285a221c726f66330cd6d08)

        • (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11371 (See https://builds.apache.org/job/Hadoop-trunk-Commit/11371/ ) HDFS-11152 . Start erasure coding policy ID number from 1 instead of 0 to (wang: rev 5addacb1e301991a8285a221c726f66330cd6d08) (edit) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/HdfsConstants.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java

          People

          • Assignee:
            Sammi SammiChen
            Reporter:
            Sammi SammiChen
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development