Colin P. McCabe added a comment - 19/Feb/16 20:51 This is a preliminary version. I need to add some unit tests. There are also some things still missing from the XML output here and there that need to be added... such as xattrs.

Colin P. McCabe added a comment - 19/Feb/16 21:21 Edit: I need to undo the effects of XMLUtils#mangleXmlString here... I haven't added that yet

Colin P. McCabe added a comment - 26/Feb/16 03:31 Add xattr encode / decode. Test xattrs with non-UTF8-compatible values.

Hadoop QA added a comment - 26/Feb/16 05:20 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 12s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 6m 55s trunk passed +1 compile 5m 57s trunk passed with JDK v1.8.0_72 +1 compile 6m 36s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 7s trunk passed +1 mvnsite 1m 52s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 26s trunk passed +1 javadoc 2m 3s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 51s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 25s the patch passed +1 compile 5m 47s the patch passed with JDK v1.8.0_72 +1 javac 5m 47s the patch passed +1 compile 6m 38s the patch passed with JDK v1.7.0_95 +1 javac 6m 38s the patch passed -1 checkstyle 1m 4s root: patch generated 114 new + 41 unchanged - 1 fixed = 155 total (was 42) +1 mvnsite 1m 52s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 2m 17s hadoop-hdfs-project/hadoop-hdfs generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) +1 javadoc 1m 58s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 52s the patch passed with JDK v1.7.0_95 +1 unit 7m 33s hadoop-common in the patch passed with JDK v1.8.0_72. -1 unit 69m 14s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 8m 55s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 74m 28s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 34s Patch does not generate ASF License warnings. 220m 27s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Load of known null value in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java: [line 670]   Load of known null value in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java: [line 667]   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1468] is not discharged   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1467] is not discharged   Exceptional return value of java.io.File.delete() ignored in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) At OfflineImageReconstructor.java:ignored in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) At OfflineImageReconstructor.java: [line 1456]   Boxing/unboxing to parse a primitive org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java:org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java: [line 953]   Boxing/unboxing to parse a primitive org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java:org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java: [line 946]   Should org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$Node be a static inner class? At OfflineImageReconstructor.java:inner class? At OfflineImageReconstructor.java: [lines 214-325] JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 JDK v1.7.0_95 Timed out junit tests org.apache.hadoop.hdfs.TestLeaseRecovery2 Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790032/HDFS-9835.002.patch JIRA Issue HDFS-9835 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux fbe5a4231e1c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d7fdec1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14620/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14620/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14620/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Hadoop QA added a comment - 26/Feb/16 09:43 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 38s Maven dependency ordering for branch +1 mvninstall 9m 40s trunk passed +1 compile 11m 47s trunk passed with JDK v1.8.0_72 +1 compile 8m 23s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 23s trunk passed +1 mvnsite 2m 46s trunk passed +1 mvneclipse 0m 34s trunk passed +1 findbugs 3m 41s trunk passed +1 javadoc 2m 24s trunk passed with JDK v1.8.0_72 +1 javadoc 3m 3s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 32s the patch passed +1 compile 8m 32s the patch passed with JDK v1.8.0_72 +1 javac 8m 32s the patch passed +1 compile 7m 51s the patch passed with JDK v1.7.0_95 +1 javac 7m 51s the patch passed -1 checkstyle 1m 10s root: patch generated 113 new + 41 unchanged - 1 fixed = 154 total (was 42) +1 mvnsite 2m 10s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 2m 25s hadoop-hdfs-project/hadoop-hdfs generated 8 new + 0 unchanged - 0 fixed = 8 total (was 0) +1 javadoc 2m 31s the patch passed with JDK v1.8.0_72 +1 javadoc 3m 10s the patch passed with JDK v1.7.0_95 -1 unit 21m 10s hadoop-common in the patch failed with JDK v1.8.0_72. -1 unit 103m 24s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 11m 34s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 84m 43s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 33s Patch does not generate ASF License warnings. 299m 38s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Load of known null value in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java: [line 673]   Load of known null value in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java: [line 670]   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1511] is not discharged   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1510] is not discharged   Exceptional return value of java.io.File.delete() ignored in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) At OfflineImageReconstructor.java:ignored in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) At OfflineImageReconstructor.java: [line 1499]   Boxing/unboxing to parse a primitive org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java:org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java: [line 996]   Boxing/unboxing to parse a primitive org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java:org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$INodeDirectorySectionProcessor.process() At OfflineImageReconstructor.java: [line 989]   Should org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$Node be a static inner class? At OfflineImageReconstructor.java:inner class? At OfflineImageReconstructor.java: [lines 217-328] JDK v1.8.0_72 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.metrics2.impl.TestGangliaMetrics   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.datanode.TestDataNodeMultipleRegistrations   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.datanode.TestBlockReplacement   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.8.0_72 Timed out junit tests org.apache.hadoop.http.TestHttpServerLifecycle JDK v1.7.0_95 Failed junit tests hadoop.hdfs.TestPersistBlocks   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.balancer.TestBalancer   hadoop.hdfs.TestSafeMode Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790058/HDFS-9835.003.patch JIRA Issue HDFS-9835 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 533763f40651 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d7fdec1 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14622/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14622/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14622/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Kihwal Lee added a comment - 26/Feb/16 16:32 - edited Are tokens and secrets/keys saved and restored?

Lei (Eddy) Xu added a comment - 26/Feb/16 21:54 Hi, Colin P. McCabe Thanks for working on it. It looks nice. A few small nitpicks: int encodedName = xattr.getName(); int ns = (0x3 & (encodedName >> 30)) | ((0x1 & (encodedName >> 5)) << 2); o( "ns" , XAttrProtos.XAttrProto. XAttrNamespaceProto.valueOf(ns).toString()); o( "name" , stringTable[0xffffff & (encodedName >> 6)]); Can we use FSImageFormatPBINode.XATTR_... constants here to make it more readable? for ( int refId : e.getRefChildrenList()) { o( "refchild" , refId); } "refchild" -> "refChild" ? Will +1 once they are addressed.

Colin P. McCabe added a comment - 26/Feb/16 22:33 Are tokens and secrets/keys saved and restored? Currently, no, but I think we can add that pretty easily. Can we use FSImageFormatPBINode.XATTR_... constants here to make it more readable? ok "refchild" -> "refChild" ? ok

Colin P. McCabe added a comment - 27/Feb/16 00:01 Fix findbugs issues Add DelegationToken serialization / deserialization support Use constants for bit shifting xattr fields

Hadoop QA added a comment - 27/Feb/16 05:02 -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 8s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 7m 31s trunk passed +1 compile 8m 36s trunk passed with JDK v1.8.0_72 +1 compile 7m 40s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 6s trunk passed +1 mvnsite 2m 2s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 41s trunk passed +1 javadoc 2m 23s trunk passed with JDK v1.8.0_72 +1 javadoc 3m 9s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 36s the patch passed +1 compile 8m 31s the patch passed with JDK v1.8.0_72 +1 javac 8m 31s the patch passed +1 compile 7m 43s the patch passed with JDK v1.7.0_95 +1 javac 7m 43s the patch passed -1 checkstyle 1m 9s root: patch generated 118 new + 57 unchanged - 1 fixed = 175 total (was 58) +1 mvnsite 2m 7s the patch passed +1 mvneclipse 0m 30s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 2m 22s hadoop-hdfs-project/hadoop-hdfs generated 7 new + 0 unchanged - 0 fixed = 7 total (was 0) +1 javadoc 2m 27s the patch passed with JDK v1.8.0_72 +1 javadoc 3m 11s the patch passed with JDK v1.7.0_95 +1 unit 10m 14s hadoop-common in the patch passed with JDK v1.8.0_72. -1 unit 84m 57s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 9m 40s hadoop-common in the patch passed with JDK v1.7.0_95. -1 unit 81m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 274m 38s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   Invocation of toString on terminators in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.loadNodeChildren(OfflineImageReconstructor$Node, String, String[]) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.loadNodeChildren(OfflineImageReconstructor$Node, String, String[]) At OfflineImageReconstructor.java: [line 405]   Load of known null value in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java:in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.processDirectoryXml(OfflineImageReconstructor$Node, FsImageProto$INodeSection$INode$Builder) At OfflineImageReconstructor.java: [line 682]   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.InputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1622] is not discharged   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.OutputStream on checked exception Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1621] is not discharged   Call to method of static java.text.DateFormat in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$SecretManagerSectionProcessor.dateStrToLong(String) At OfflineImageReconstructor.java:java.text.DateFormat in org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor$SecretManagerSectionProcessor.dateStrToLong(String) At OfflineImageReconstructor.java: [line 900]   Call to method of static java.text.DateFormat in org.apache.hadoop.hdfs.tools.offlineImageViewer.PBImageXmlWriter.dumpDate(String, long) At PBImageXmlWriter.java:java.text.DateFormat in org.apache.hadoop.hdfs.tools.offlineImageViewer.PBImageXmlWriter.dumpDate(String, long) At PBImageXmlWriter.java: [line 459]   org.apache.hadoop.hdfs.tools.offlineImageViewer.PBImageXmlWriter.ISO_DATE_FORMAT is a static field of type java.text.DateFormat, which isn't thread safe In PBImageXmlWriter.java:of type java.text.DateFormat, which isn't thread safe In PBImageXmlWriter.java JDK v1.8.0_72 Failed junit tests hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.TestFileAppend   hadoop.hdfs.TestReadStripedFileWithDecoding   hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_95 Failed junit tests hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790256/HDFS-9835.004.patch JIRA Issue HDFS-9835 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux e7e58fb8e92e 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / d1d4e16 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14635/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14635/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14635/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Colin P. McCabe added a comment - 29/Feb/16 21:36 Fix findbugs issues

Hadoop QA added a comment - 01/Mar/16 00:52 -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 49s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 1m 14s Maven dependency ordering for branch +1 mvninstall 6m 38s trunk passed +1 compile 5m 40s trunk passed with JDK v1.8.0_72 +1 compile 6m 46s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 5s trunk passed +1 mvnsite 1m 56s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 3m 25s trunk passed +1 javadoc 2m 0s trunk passed with JDK v1.8.0_72 +1 javadoc 2m 47s trunk passed with JDK v1.7.0_95 0 mvndep 0m 15s Maven dependency ordering for patch +1 mvninstall 1m 26s the patch passed +1 compile 5m 45s the patch passed with JDK v1.8.0_72 +1 javac 5m 45s the patch passed +1 compile 6m 41s the patch passed with JDK v1.7.0_95 +1 javac 6m 41s the patch passed -1 checkstyle 1m 6s root: patch generated 118 new + 57 unchanged - 1 fixed = 175 total (was 58) +1 mvnsite 1m 53s the patch passed +1 mvneclipse 0m 28s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. -1 findbugs 2m 15s hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) +1 javadoc 1m 59s the patch passed with JDK v1.8.0_72 +1 javadoc 2m 50s the patch passed with JDK v1.7.0_95 -1 unit 6m 39s hadoop-common in the patch failed with JDK v1.8.0_72. -1 unit 53m 52s hadoop-hdfs in the patch failed with JDK v1.8.0_72. +1 unit 7m 24s hadoop-common in the patch passed with JDK v1.7.0_95. +1 unit 51m 59s hadoop-hdfs in the patch passed with JDK v1.7.0_95. +1 asflicense 0m 25s Patch does not generate ASF License warnings. 194m 10s Reason Tests FindBugs module:hadoop-hdfs-project/hadoop-hdfs   org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageReconstructor.run(String, String) may fail to clean up java.io.InputStream Obligation to clean up resource created at OfflineImageReconstructor.java:clean up java.io.InputStream Obligation to clean up resource created at OfflineImageReconstructor.java: [line 1630] is not discharged JDK v1.8.0_72 Failed junit tests hadoop.metrics2.impl.TestGangliaMetrics   hadoop.hdfs.web.TestWebHDFS Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12790571/HDFS-9835.005.patch JIRA Issue HDFS-9835 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux ffbf36040a06 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 0fa54d4 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/new-findbugs-hadoop-hdfs-project_hadoop-hdfs.html unit https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14660/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14660/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14660/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Lei (Eddy) Xu added a comment - 02/Mar/16 00:40 Hi, Colin P. McCabe The patch looks good to me overall. +1 after addressing final nitpicks: There are some checkstyle warnings for switch...case indentions. The findbugs warning is false, but can we mitigate it somehow? Could you also apply XATTR_..._MASKs to PBImageXmlWriter#dumpXAttrs . Thanks a lot for the work.

Colin P. McCabe added a comment - 02/Mar/16 19:40 Thanks, Lei (Eddy) Xu . Reposted v6

Lei (Eddy) Xu added a comment - 02/Mar/16 20:24 Thanks a lot, Colin P. McCabe . +1. Pending Jenkins.

Hadoop QA added a comment - 03/Mar/16 01:18 -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 8m 11s trunk passed +1 compile 11m 16s trunk passed with JDK v1.8.0_72 +1 compile 8m 32s trunk passed with JDK v1.7.0_95 +1 checkstyle 1m 10s trunk passed +1 mvnsite 2m 14s trunk passed +1 mvneclipse 0m 28s trunk passed +1 findbugs 4m 5s trunk passed +1 javadoc 2m 31s trunk passed with JDK v1.8.0_72 +1 javadoc 3m 39s trunk passed with JDK v1.7.0_95 0 mvndep 0m 17s Maven dependency ordering for patch +1 mvninstall 2m 1s the patch passed +1 compile 10m 23s the patch passed with JDK v1.8.0_72 +1 javac 10m 23s the patch passed +1 compile 8m 24s the patch passed with JDK v1.7.0_95 +1 javac 8m 24s the patch passed -1 checkstyle 1m 11s root: patch generated 29 new + 58 unchanged - 1 fixed = 87 total (was 59) +1 mvnsite 2m 15s the patch passed +1 mvneclipse 0m 30s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 findbugs 4m 24s the patch passed +1 javadoc 2m 27s the patch passed with JDK v1.8.0_72 +1 javadoc 3m 45s the patch passed with JDK v1.7.0_95 -1 unit 10m 38s hadoop-common in the patch failed with JDK v1.8.0_72. -1 unit 84m 31s hadoop-hdfs in the patch failed with JDK v1.8.0_72. -1 unit 9m 14s hadoop-common in the patch failed with JDK v1.7.0_95. -1 unit 65m 33s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 249m 53s Reason Tests JDK v1.8.0_72 Failed junit tests hadoop.ipc.TestRPCWaitForProxy   hadoop.ha.TestZKFailoverController   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.qjournal.TestSecureNNWithQJM   hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider   hadoop.hdfs.TestSafeModeWithStripedFile JDK v1.7.0_95 Failed junit tests hadoop.ha.TestZKFailoverController   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.blockmanagement.TestBlockManager Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12791008/HDFS-9835.006.patch JIRA Issue HDFS-9835 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 9f181c803775 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 67880cc Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_72 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_72.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_95.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14693/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14693/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14693/console Powered by Apache Yetus 0.3.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.

Hudson added a comment - 03/Mar/16 02:06 FAILURE: Integrated in Hadoop-trunk-Commit #9414 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9414/ ) HDFS-9835 . OIV: add ReverseXML processor which reconstructs an fsimage (cmccabe: rev 700b0e4019cf483f7532609711812150b8c44742) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageViewerPB.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/test/GenericTestUtils.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/PBImageXmlWriter.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageFormatPBINode.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/TestOfflineImageViewer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/OfflineImageReconstructor.java

Colin P. McCabe added a comment - 03/Mar/16 02:19 - edited committed to 2.8