Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13830

Backport HDFS-13141 to branch-3.0: WebHDFS: Add support for getting snasphottable directory list

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.0.3
    • 3.0.4
    • webhdfs
    • None

    Description

      HDFS-13141 conflicts with 3.0.3 because of interface change in HdfsFileStatus.

      This Jira aims to backport the WebHDFS getSnapshottableDirListing() support to branch-3.0.

      Attachments

        1. HDFS-13830.branch-3.0.001.patch
          16 kB
          Siyao Meng
        2. HDFS-13830.branch-3.0.002.patch
          16 kB
          Siyao Meng
        3. HDFS-13830.branch-3.0.003.patch
          18 kB
          Siyao Meng
        4. HDFS-13830.branch-3.0.004.patch
          18 kB
          Siyao Meng

        Issue Links

          Activity

            smeng Siyao Meng added a comment -

            Found function name change in HDFS-12681. I kept the old function name flags() but added the new param.

            Patch rev 001 submitted.

            smeng Siyao Meng added a comment - Found function name change in HDFS-12681 . I kept the old function name flags() but added the new param. Patch rev 001 submitted.
            genericqa genericqa added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 17m 50s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  branch-3.0 Compile Tests
            0 mvndep 4m 14s Maven dependency ordering for branch
            +1 mvninstall 18m 36s branch-3.0 passed
            +1 compile 12m 28s branch-3.0 passed
            +1 checkstyle 2m 39s branch-3.0 passed
            +1 mvnsite 2m 43s branch-3.0 passed
            +1 shadedclient 15m 6s branch has no errors when building and testing our client artifacts.
            +1 findbugs 4m 54s branch-3.0 passed
            +1 javadoc 2m 18s branch-3.0 passed
                  Patch Compile Tests
            0 mvndep 0m 18s Maven dependency ordering for patch
            +1 mvninstall 2m 13s the patch passed
            +1 compile 12m 6s the patch passed
            -1 javac 12m 6s root generated 2 new + 1251 unchanged - 0 fixed = 1253 total (was 1251)
            -0 checkstyle 2m 40s root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274)
            +1 mvnsite 3m 1s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 9m 51s patch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 34s the patch passed
            +1 javadoc 2m 20s the patch passed
                  Other Tests
            +1 unit 8m 1s hadoop-common in the patch passed.
            +1 unit 1m 32s hadoop-hdfs-client in the patch passed.
            -1 unit 98m 27s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 49s The patch does not generate ASF License warnings.
            224m 55s



            Reason Tests
            Failed junit tests hadoop.hdfs.web.TestWebHDFS
              hadoop.hdfs.server.datanode.TestDirectoryScanner
              hadoop.hdfs.web.TestWebHdfsTimeouts
              hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208
            JIRA Issue HDFS-13830
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936076/HDFS-13830.branch-3.0.001.patch
            Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 994f5fde58ae 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-3.0 / a41f180
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            javac https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-compile-javac-root.txt
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24802/testReport/
            Max. process+thread count 4283 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24802/console
            Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 50s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       branch-3.0 Compile Tests 0 mvndep 4m 14s Maven dependency ordering for branch +1 mvninstall 18m 36s branch-3.0 passed +1 compile 12m 28s branch-3.0 passed +1 checkstyle 2m 39s branch-3.0 passed +1 mvnsite 2m 43s branch-3.0 passed +1 shadedclient 15m 6s branch has no errors when building and testing our client artifacts. +1 findbugs 4m 54s branch-3.0 passed +1 javadoc 2m 18s branch-3.0 passed       Patch Compile Tests 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 2m 13s the patch passed +1 compile 12m 6s the patch passed -1 javac 12m 6s root generated 2 new + 1251 unchanged - 0 fixed = 1253 total (was 1251) -0 checkstyle 2m 40s root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274) +1 mvnsite 3m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 9m 51s patch has no errors when building and testing our client artifacts. +1 findbugs 5m 34s the patch passed +1 javadoc 2m 20s the patch passed       Other Tests +1 unit 8m 1s hadoop-common in the patch passed. +1 unit 1m 32s hadoop-hdfs-client in the patch passed. -1 unit 98m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 49s The patch does not generate ASF License warnings. 224m 55s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHDFS   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 JIRA Issue HDFS-13830 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936076/HDFS-13830.branch-3.0.001.patch Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 994f5fde58ae 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-3.0 / a41f180 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 javac https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-compile-javac-root.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24802/testReport/ Max. process+thread count 4283 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24802/console Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            smeng Siyao Meng added a comment -

            Fixing FileStatus.isDir() deprecated warning in TestWebHDFS.java.

            Won't fix checkstyle "public enum AttrFlags" javadoc warning since it is not related to this Jira and it has been added in branch-3.1.0 by HDFS-12681.

            Won't fix checkstyle "Avoid nested blocks" warning because the rest of the code in the switch case are also written in this "nested" style.

            smeng Siyao Meng added a comment - Fixing FileStatus.isDir() deprecated warning in TestWebHDFS.java. Won't fix checkstyle "public enum AttrFlags" javadoc warning since it is not related to this Jira and it has been added in branch-3.1.0 by HDFS-12681 . Won't fix checkstyle "Avoid nested blocks" warning because the rest of the code in the switch case are also written in this "nested" style.
            genericqa genericqa added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 15m 33s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  branch-3.0 Compile Tests
            0 mvndep 2m 12s Maven dependency ordering for branch
            +1 mvninstall 19m 30s branch-3.0 passed
            +1 compile 18m 14s branch-3.0 passed
            +1 checkstyle 3m 52s branch-3.0 passed
            +1 mvnsite 3m 58s branch-3.0 passed
            +1 shadedclient 20m 5s branch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 28s branch-3.0 passed
            +1 javadoc 2m 29s branch-3.0 passed
                  Patch Compile Tests
            0 mvndep 0m 17s Maven dependency ordering for patch
            +1 mvninstall 2m 20s the patch passed
            +1 compile 15m 8s the patch passed
            +1 javac 15m 8s the patch passed
            -0 checkstyle 3m 22s root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274)
            +1 mvnsite 3m 42s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 11m 51s patch has no errors when building and testing our client artifacts.
            +1 findbugs 6m 27s the patch passed
            +1 javadoc 2m 45s the patch passed
                  Other Tests
            +1 unit 9m 20s hadoop-common in the patch passed.
            +1 unit 1m 35s hadoop-hdfs-client in the patch passed.
            -1 unit 89m 35s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 42s The patch does not generate ASF License warnings.
            233m 40s



            Reason Tests
            Failed junit tests hadoop.hdfs.TestFileAppend
              hadoop.hdfs.web.TestWebHDFS



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208
            JIRA Issue HDFS-13830
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936539/HDFS-13830.branch-3.0.002.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux f2cd442b7371 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-3.0 / bd46906
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24829/testReport/
            Max. process+thread count 3246 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24829/console
            Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 15m 33s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       branch-3.0 Compile Tests 0 mvndep 2m 12s Maven dependency ordering for branch +1 mvninstall 19m 30s branch-3.0 passed +1 compile 18m 14s branch-3.0 passed +1 checkstyle 3m 52s branch-3.0 passed +1 mvnsite 3m 58s branch-3.0 passed +1 shadedclient 20m 5s branch has no errors when building and testing our client artifacts. +1 findbugs 5m 28s branch-3.0 passed +1 javadoc 2m 29s branch-3.0 passed       Patch Compile Tests 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 20s the patch passed +1 compile 15m 8s the patch passed +1 javac 15m 8s the patch passed -0 checkstyle 3m 22s root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274) +1 mvnsite 3m 42s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 11m 51s patch has no errors when building and testing our client artifacts. +1 findbugs 6m 27s the patch passed +1 javadoc 2m 45s the patch passed       Other Tests +1 unit 9m 20s hadoop-common in the patch passed. +1 unit 1m 35s hadoop-hdfs-client in the patch passed. -1 unit 89m 35s hadoop-hdfs in the patch failed. +1 asflicense 0m 42s The patch does not generate ASF License warnings. 233m 40s Reason Tests Failed junit tests hadoop.hdfs.TestFileAppend   hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 JIRA Issue HDFS-13830 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936539/HDFS-13830.branch-3.0.002.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux f2cd442b7371 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-3.0 / bd46906 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24829/testReport/ Max. process+thread count 3246 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24829/console Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            smeng Siyao Meng added a comment -

            rev 003 changelog:

            1. Added missing pieces in JsonUtil.
            2. Integrated HDFS-13280 NPE fix.
            smeng Siyao Meng added a comment - rev 003 changelog: Added missing pieces in JsonUtil. Integrated HDFS-13280 NPE fix.
            genericqa genericqa added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 16m 36s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  branch-3.0 Compile Tests
            0 mvndep 2m 15s Maven dependency ordering for branch
            +1 mvninstall 18m 9s branch-3.0 passed
            +1 compile 11m 51s branch-3.0 passed
            +1 checkstyle 2m 40s branch-3.0 passed
            +1 mvnsite 2m 47s branch-3.0 passed
            +1 shadedclient 15m 36s branch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 7s branch-3.0 passed
            +1 javadoc 2m 14s branch-3.0 passed
                  Patch Compile Tests
            0 mvndep 0m 17s Maven dependency ordering for patch
            +1 mvninstall 2m 14s the patch passed
            +1 compile 11m 13s the patch passed
            +1 javac 11m 13s the patch passed
            -0 checkstyle 2m 56s root: The patch generated 2 new + 282 unchanged - 2 fixed = 284 total (was 284)
            +1 mvnsite 2m 51s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 9m 14s patch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 22s the patch passed
            +1 javadoc 2m 7s the patch passed
                  Other Tests
            -1 unit 8m 11s hadoop-common in the patch failed.
            +1 unit 1m 42s hadoop-hdfs-client in the patch passed.
            -1 unit 89m 27s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 45s The patch does not generate ASF License warnings.
            210m 33s



            Reason Tests
            Failed junit tests hadoop.ha.TestZKFailoverController
              hadoop.hdfs.server.balancer.TestBalancer
              hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
              hadoop.hdfs.web.TestWebHdfsTimeouts
              hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208
            JIRA Issue HDFS-13830
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936901/HDFS-13830.branch-3.0.003.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 505248a1a1ab 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-3.0 / 79ebbec
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24858/testReport/
            Max. process+thread count 4937 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24858/console
            Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 16m 36s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       branch-3.0 Compile Tests 0 mvndep 2m 15s Maven dependency ordering for branch +1 mvninstall 18m 9s branch-3.0 passed +1 compile 11m 51s branch-3.0 passed +1 checkstyle 2m 40s branch-3.0 passed +1 mvnsite 2m 47s branch-3.0 passed +1 shadedclient 15m 36s branch has no errors when building and testing our client artifacts. +1 findbugs 5m 7s branch-3.0 passed +1 javadoc 2m 14s branch-3.0 passed       Patch Compile Tests 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 14s the patch passed +1 compile 11m 13s the patch passed +1 javac 11m 13s the patch passed -0 checkstyle 2m 56s root: The patch generated 2 new + 282 unchanged - 2 fixed = 284 total (was 284) +1 mvnsite 2m 51s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 9m 14s patch has no errors when building and testing our client artifacts. +1 findbugs 5m 22s the patch passed +1 javadoc 2m 7s the patch passed       Other Tests -1 unit 8m 11s hadoop-common in the patch failed. +1 unit 1m 42s hadoop-hdfs-client in the patch passed. -1 unit 89m 27s hadoop-hdfs in the patch failed. +1 asflicense 0m 45s The patch does not generate ASF License warnings. 210m 33s Reason Tests Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 JIRA Issue HDFS-13830 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12936901/HDFS-13830.branch-3.0.003.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 505248a1a1ab 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-3.0 / 79ebbec maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24858/testReport/ Max. process+thread count 4937 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24858/console Powered by Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            smeng Siyao Meng added a comment -

            All failed tests passed locally. Won't fix checkstyle for reason mentioned above.

            smeng Siyao Meng added a comment - All failed tests passed locally. Won't fix checkstyle for reason mentioned above.

            smeng thanks for the rev003 patch.

            1. Added missing pieces in JsonUtil.
            2. Integrated HDFS-13280 NPE fix.

            Please refrain from incorporating fixes into the same patch, because it makes diagnosing problems harder in the future. Instead, we should backport HDFS-13280 to branch 3.0 on its own.

            weichiu Wei-Chiu Chuang added a comment - smeng thanks for the rev003 patch. Added missing pieces in JsonUtil. Integrated  HDFS-13280  NPE fix. Please refrain from incorporating fixes into the same patch, because it makes diagnosing problems harder in the future. Instead, we should backport HDFS-13280 to branch 3.0 on its own.
            smeng Siyao Meng added a comment - - edited

            Thanks weichiu for the comment.

            Removed HDFS-13280 patch in rev 004.

            smeng Siyao Meng added a comment - - edited Thanks  weichiu for the comment. Removed HDFS-13280 patch in rev 004.
            genericqa genericqa added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 17m 0s Docker mode activated.
                  Prechecks
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
                  branch-3.0 Compile Tests
            0 mvndep 2m 15s Maven dependency ordering for branch
            +1 mvninstall 18m 18s branch-3.0 passed
            +1 compile 12m 12s branch-3.0 passed
            +1 checkstyle 2m 44s branch-3.0 passed
            +1 mvnsite 3m 5s branch-3.0 passed
            +1 shadedclient 16m 18s branch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 16s branch-3.0 passed
            +1 javadoc 2m 34s branch-3.0 passed
                  Patch Compile Tests
            0 mvndep 0m 18s Maven dependency ordering for patch
            +1 mvninstall 2m 20s the patch passed
            +1 compile 11m 19s the patch passed
            +1 javac 11m 19s the patch passed
            -0 checkstyle 2m 43s root: The patch generated 2 new + 281 unchanged - 2 fixed = 283 total (was 283)
            +1 mvnsite 3m 2s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 shadedclient 9m 35s patch has no errors when building and testing our client artifacts.
            +1 findbugs 5m 27s the patch passed
            +1 javadoc 2m 25s the patch passed
                  Other Tests
            +1 unit 8m 0s hadoop-common in the patch passed.
            +1 unit 1m 38s hadoop-hdfs-client in the patch passed.
            -1 unit 93m 21s hadoop-hdfs in the patch failed.
            +1 asflicense 0m 44s The patch does not generate ASF License warnings.
            217m 30s



            Reason Tests
            Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts
              hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints
              hadoop.hdfs.TestLeaseRecovery2



            Subsystem Report/Notes
            Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208
            JIRA Issue HDFS-13830
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12937128/HDFS-13830.branch-3.0.004.patch
            Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
            uname Linux 36d705a03007 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/provided.sh
            git revision branch-3.0 / 27abdb3
            maven version: Apache Maven 3.3.9
            Default Java 1.8.0_181
            findbugs v3.1.0-RC1
            checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/diff-checkstyle-root.txt
            unit https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
            Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24884/testReport/
            Max. process+thread count 5004 (vs. ulimit of 10000)
            modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
            Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24884/console
            Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org

            This message was automatically generated.

            genericqa genericqa added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 0s Docker mode activated.       Prechecks +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.       branch-3.0 Compile Tests 0 mvndep 2m 15s Maven dependency ordering for branch +1 mvninstall 18m 18s branch-3.0 passed +1 compile 12m 12s branch-3.0 passed +1 checkstyle 2m 44s branch-3.0 passed +1 mvnsite 3m 5s branch-3.0 passed +1 shadedclient 16m 18s branch has no errors when building and testing our client artifacts. +1 findbugs 5m 16s branch-3.0 passed +1 javadoc 2m 34s branch-3.0 passed       Patch Compile Tests 0 mvndep 0m 18s Maven dependency ordering for patch +1 mvninstall 2m 20s the patch passed +1 compile 11m 19s the patch passed +1 javac 11m 19s the patch passed -0 checkstyle 2m 43s root: The patch generated 2 new + 281 unchanged - 2 fixed = 283 total (was 283) +1 mvnsite 3m 2s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 shadedclient 9m 35s patch has no errors when building and testing our client artifacts. +1 findbugs 5m 27s the patch passed +1 javadoc 2m 25s the patch passed       Other Tests +1 unit 8m 0s hadoop-common in the patch passed. +1 unit 1m 38s hadoop-hdfs-client in the patch passed. -1 unit 93m 21s hadoop-hdfs in the patch failed. +1 asflicense 0m 44s The patch does not generate ASF License warnings. 217m 30s Reason Tests Failed junit tests hadoop.hdfs.web.TestWebHdfsTimeouts   hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints   hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 JIRA Issue HDFS-13830 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12937128/HDFS-13830.branch-3.0.004.patch Optional Tests dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle uname Linux 36d705a03007 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/provided.sh git revision branch-3.0 / 27abdb3 maven version: Apache Maven 3.3.9 Default Java 1.8.0_181 findbugs v3.1.0-RC1 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/24884/testReport/ Max. process+thread count 5004 (vs. ulimit of 10000) modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/24884/console Powered by Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
            smeng Siyao Meng added a comment -

            +1 jenkins. Failed tests passed locally.

            smeng Siyao Meng added a comment - +1 jenkins. Failed tests passed locally.

            The patch looks mostly good. There are two extra lines in the test though:

            TestWebHDFS#testWebHdfsSnapshottableDirectoryList
            	SnapshottableDirectoryStatus[] statuses =
            	webHdfs.getSnapshottableDirectoryList();
            	Assert.assertNull(statuses);
            weichiu Wei-Chiu Chuang added a comment - The patch looks mostly good. There are two extra lines in the test though: TestWebHDFS#testWebHdfsSnapshottableDirectoryList SnapshottableDirectoryStatus[] statuses = webHdfs.getSnapshottableDirectoryList(); Assert.assertNull(statuses);
            smeng Siyao Meng added a comment - - edited

            weichiu Thanks for the review!
            But those two lines are already removed in rev 004.

            smeng Siyao Meng added a comment - - edited weichiu Thanks for the review! But those two lines are already removed in rev 004.

            +1 

            Thanks smeng

            Also checked compat guide line  since this patch changes the visibility of methods and static class fields of FileStatus, which is widely used and Public.Stable, it is essentially a compatible change. Under Hadoop compatibility guideline http://hadoop.apache.org/docs/r3.0.3/hadoop-project-dist/hadoop-common/InterfaceClassification.html#Stable, a compat change can be made between maintenance releases. (Note there's a typo being addressed by HADOOP-15705)

            (Wish Yetus could detect Hadoop compatibility violation)

            weichiu Wei-Chiu Chuang added a comment - +1  Thanks  smeng .  Also checked compat guide line  since this patch changes the visibility of methods and static class fields of FileStatus, which is widely used and Public.Stable, it is essentially a compatible change. Under Hadoop compatibility guideline  http://hadoop.apache.org/docs/r3.0.3/hadoop-project-dist/hadoop-common/InterfaceClassification.html#Stable,  a compat change can be made between maintenance releases. (Note there's a typo being addressed by  HADOOP-15705 ) (Wish Yetus could detect Hadoop compatibility violation)

            Actually, the typo goes the other direction. A compatible change is always allowed in a maintenance release, no matter what the stability label. It's compatible, so it's not an issue.

            templedf Daniel Templeton added a comment - Actually, the typo goes the other direction. A compatible change is always allowed in a maintenance release, no matter what the stability label. It's compatible, so it's not an issue.
            smeng Siyao Meng added a comment - - edited

            Thanks weichiu for the review. And thanks templedf for the comment.

            I believe this patch is ready for commit.

            smeng Siyao Meng added a comment - - edited Thanks weichiu for the review. And thanks templedf for the comment. I believe this patch is ready for commit.

            +1 will commit later today

            weichiu Wei-Chiu Chuang added a comment - +1 will commit later today

            Pushed to branch-3.0. Thanks smeng!

            weichiu Wei-Chiu Chuang added a comment - Pushed to branch-3.0. Thanks smeng !
            smeng Siyao Meng added a comment - - edited

            weichiu Thanks for reviewing and committing!

            smeng Siyao Meng added a comment - - edited weichiu Thanks for reviewing and committing!

            People

              smeng Siyao Meng
              smeng Siyao Meng
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: