Description
HDFS-13141 conflicts with 3.0.3 because of interface change in HdfsFileStatus.
This Jira aims to backport the WebHDFS getSnapshottableDirListing() support to branch-3.0.
Attachments
Attachments
- HDFS-13830.branch-3.0.001.patch
- 16 kB
- Siyao Meng
- HDFS-13830.branch-3.0.002.patch
- 16 kB
- Siyao Meng
- HDFS-13830.branch-3.0.003.patch
- 18 kB
- Siyao Meng
- HDFS-13830.branch-3.0.004.patch
- 18 kB
- Siyao Meng
Issue Links
- Blocked
-
HDFS-13280 WebHDFS: Fix NPE in get snasphottable directory list call
- Resolved
- causes
-
HDFS-14009 HttpFS: FileStatus#setSnapShotEnabledFlag throws InvocationTargetException when attribute set is emptySet
- Resolved
- is part of
-
HDFS-13141 WebHDFS: Add support for getting snasphottable directory list
- Resolved
- relates to
-
HDFS-13838 WebHdfsFileSystem.getFileStatus() won't return correct "snapshot enabled" status
- Resolved
Activity
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 17m 50s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
branch-3.0 Compile Tests | |||
0 | mvndep | 4m 14s | Maven dependency ordering for branch |
+1 | mvninstall | 18m 36s | branch-3.0 passed |
+1 | compile | 12m 28s | branch-3.0 passed |
+1 | checkstyle | 2m 39s | branch-3.0 passed |
+1 | mvnsite | 2m 43s | branch-3.0 passed |
+1 | shadedclient | 15m 6s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 4m 54s | branch-3.0 passed |
+1 | javadoc | 2m 18s | branch-3.0 passed |
Patch Compile Tests | |||
0 | mvndep | 0m 18s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 13s | the patch passed |
+1 | compile | 12m 6s | the patch passed |
-1 | javac | 12m 6s | root generated 2 new + 1251 unchanged - 0 fixed = 1253 total (was 1251) |
-0 | checkstyle | 2m 40s | root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274) |
+1 | mvnsite | 3m 1s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 9m 51s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 34s | the patch passed |
+1 | javadoc | 2m 20s | the patch passed |
Other Tests | |||
+1 | unit | 8m 1s | hadoop-common in the patch passed. |
+1 | unit | 1m 32s | hadoop-hdfs-client in the patch passed. |
-1 | unit | 98m 27s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 49s | The patch does not generate ASF License warnings. |
224m 55s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.web.TestWebHDFS |
hadoop.hdfs.server.datanode.TestDirectoryScanner | |
hadoop.hdfs.web.TestWebHdfsTimeouts | |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12936076/HDFS-13830.branch-3.0.001.patch |
Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux 994f5fde58ae 4.4.0-130-generic #156-Ubuntu SMP Thu Jun 14 08:53:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | branch-3.0 / a41f180 |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC1 |
javac | https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-compile-javac-root.txt |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/diff-checkstyle-root.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/24802/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/24802/testReport/ |
Max. process+thread count | 4283 (vs. ulimit of 10000) |
modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/24802/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
Fixing FileStatus.isDir() deprecated warning in TestWebHDFS.java.
Won't fix checkstyle "public enum AttrFlags" javadoc warning since it is not related to this Jira and it has been added in branch-3.1.0 by HDFS-12681.
Won't fix checkstyle "Avoid nested blocks" warning because the rest of the code in the switch case are also written in this "nested" style.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 15m 33s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
branch-3.0 Compile Tests | |||
0 | mvndep | 2m 12s | Maven dependency ordering for branch |
+1 | mvninstall | 19m 30s | branch-3.0 passed |
+1 | compile | 18m 14s | branch-3.0 passed |
+1 | checkstyle | 3m 52s | branch-3.0 passed |
+1 | mvnsite | 3m 58s | branch-3.0 passed |
+1 | shadedclient | 20m 5s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 28s | branch-3.0 passed |
+1 | javadoc | 2m 29s | branch-3.0 passed |
Patch Compile Tests | |||
0 | mvndep | 0m 17s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 20s | the patch passed |
+1 | compile | 15m 8s | the patch passed |
+1 | javac | 15m 8s | the patch passed |
-0 | checkstyle | 3m 22s | root: The patch generated 2 new + 272 unchanged - 2 fixed = 274 total (was 274) |
+1 | mvnsite | 3m 42s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 11m 51s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 6m 27s | the patch passed |
+1 | javadoc | 2m 45s | the patch passed |
Other Tests | |||
+1 | unit | 9m 20s | hadoop-common in the patch passed. |
+1 | unit | 1m 35s | hadoop-hdfs-client in the patch passed. |
-1 | unit | 89m 35s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 42s | The patch does not generate ASF License warnings. |
233m 40s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.TestFileAppend |
hadoop.hdfs.web.TestWebHDFS |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12936539/HDFS-13830.branch-3.0.002.patch |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux f2cd442b7371 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | branch-3.0 / bd46906 |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC1 |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/diff-checkstyle-root.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/24829/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/24829/testReport/ |
Max. process+thread count | 3246 (vs. ulimit of 10000) |
modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/24829/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
rev 003 changelog:
- Added missing pieces in JsonUtil.
- Integrated
HDFS-13280NPE fix.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 16m 36s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
branch-3.0 Compile Tests | |||
0 | mvndep | 2m 15s | Maven dependency ordering for branch |
+1 | mvninstall | 18m 9s | branch-3.0 passed |
+1 | compile | 11m 51s | branch-3.0 passed |
+1 | checkstyle | 2m 40s | branch-3.0 passed |
+1 | mvnsite | 2m 47s | branch-3.0 passed |
+1 | shadedclient | 15m 36s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 7s | branch-3.0 passed |
+1 | javadoc | 2m 14s | branch-3.0 passed |
Patch Compile Tests | |||
0 | mvndep | 0m 17s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 14s | the patch passed |
+1 | compile | 11m 13s | the patch passed |
+1 | javac | 11m 13s | the patch passed |
-0 | checkstyle | 2m 56s | root: The patch generated 2 new + 282 unchanged - 2 fixed = 284 total (was 284) |
+1 | mvnsite | 2m 51s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 9m 14s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 22s | the patch passed |
+1 | javadoc | 2m 7s | the patch passed |
Other Tests | |||
-1 | unit | 8m 11s | hadoop-common in the patch failed. |
+1 | unit | 1m 42s | hadoop-hdfs-client in the patch passed. |
-1 | unit | 89m 27s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 45s | The patch does not generate ASF License warnings. |
210m 33s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.ha.TestZKFailoverController |
hadoop.hdfs.server.balancer.TestBalancer | |
hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits | |
hadoop.hdfs.web.TestWebHdfsTimeouts | |
hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12936901/HDFS-13830.branch-3.0.003.patch |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux 505248a1a1ab 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | branch-3.0 / 79ebbec |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC1 |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/diff-checkstyle-root.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/24858/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/24858/testReport/ |
Max. process+thread count | 4937 (vs. ulimit of 10000) |
modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/24858/console |
Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
All failed tests passed locally. Won't fix checkstyle for reason mentioned above.
smeng thanks for the rev003 patch.
- Added missing pieces in JsonUtil.
- Integrated
HDFS-13280NPE fix.
Please refrain from incorporating fixes into the same patch, because it makes diagnosing problems harder in the future. Instead, we should backport HDFS-13280 to branch 3.0 on its own.
-1 overall |
Vote | Subsystem | Runtime | Comment |
---|---|---|---|
0 | reexec | 17m 0s | Docker mode activated. |
Prechecks | |||
+1 | @author | 0m 0s | The patch does not contain any @author tags. |
+1 | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. |
branch-3.0 Compile Tests | |||
0 | mvndep | 2m 15s | Maven dependency ordering for branch |
+1 | mvninstall | 18m 18s | branch-3.0 passed |
+1 | compile | 12m 12s | branch-3.0 passed |
+1 | checkstyle | 2m 44s | branch-3.0 passed |
+1 | mvnsite | 3m 5s | branch-3.0 passed |
+1 | shadedclient | 16m 18s | branch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 16s | branch-3.0 passed |
+1 | javadoc | 2m 34s | branch-3.0 passed |
Patch Compile Tests | |||
0 | mvndep | 0m 18s | Maven dependency ordering for patch |
+1 | mvninstall | 2m 20s | the patch passed |
+1 | compile | 11m 19s | the patch passed |
+1 | javac | 11m 19s | the patch passed |
-0 | checkstyle | 2m 43s | root: The patch generated 2 new + 281 unchanged - 2 fixed = 283 total (was 283) |
+1 | mvnsite | 3m 2s | the patch passed |
+1 | whitespace | 0m 0s | The patch has no whitespace issues. |
+1 | shadedclient | 9m 35s | patch has no errors when building and testing our client artifacts. |
+1 | findbugs | 5m 27s | the patch passed |
+1 | javadoc | 2m 25s | the patch passed |
Other Tests | |||
+1 | unit | 8m 0s | hadoop-common in the patch passed. |
+1 | unit | 1m 38s | hadoop-hdfs-client in the patch passed. |
-1 | unit | 93m 21s | hadoop-hdfs in the patch failed. |
+1 | asflicense | 0m 44s | The patch does not generate ASF License warnings. |
217m 30s |
Reason | Tests |
---|---|
Failed junit tests | hadoop.hdfs.web.TestWebHdfsTimeouts |
hadoop.hdfs.server.namenode.ha.TestStandbyCheckpoints | |
hadoop.hdfs.TestLeaseRecovery2 |
Subsystem | Report/Notes |
---|---|
Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:1776208 |
JIRA Issue | |
JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12937128/HDFS-13830.branch-3.0.004.patch |
Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle |
uname | Linux 36d705a03007 4.4.0-133-generic #159-Ubuntu SMP Fri Aug 10 07:31:43 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
Build tool | maven |
Personality | /testptch/patchprocess/precommit/personality/provided.sh |
git revision | branch-3.0 / 27abdb3 |
maven | version: Apache Maven 3.3.9 |
Default Java | 1.8.0_181 |
findbugs | v3.1.0-RC1 |
checkstyle | https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/diff-checkstyle-root.txt |
unit | https://builds.apache.org/job/PreCommit-HDFS-Build/24884/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt |
Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/24884/testReport/ |
Max. process+thread count | 5004 (vs. ulimit of 10000) |
modules | C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . |
Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/24884/console |
Powered by | Apache Yetus 0.9.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
The patch looks mostly good. There are two extra lines in the test though:
SnapshottableDirectoryStatus[] statuses = webHdfs.getSnapshottableDirectoryList(); Assert.assertNull(statuses);
+1
Thanks smeng.
Also checked compat guide line since this patch changes the visibility of methods and static class fields of FileStatus, which is widely used and Public.Stable, it is essentially a compatible change. Under Hadoop compatibility guideline http://hadoop.apache.org/docs/r3.0.3/hadoop-project-dist/hadoop-common/InterfaceClassification.html#Stable, a compat change can be made between maintenance releases. (Note there's a typo being addressed by HADOOP-15705)
(Wish Yetus could detect Hadoop compatibility violation)
Actually, the typo goes the other direction. A compatible change is always allowed in a maintenance release, no matter what the stability label. It's compatible, so it's not an issue.
Thanks weichiu for the review. And thanks templedf for the comment.
I believe this patch is ready for commit.
Found function name change in
HDFS-12681. I kept the old function name flags() but added the new param.Patch rev 001 submitted.