Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8898

Create API and command-line argument to get quota and quota usage without detailed content summary

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: fs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      On large directory structures it takes significant time to iterate through the file and directory counts recursively to get a complete ContentSummary.
      When you want to just check for the quota on a higher level directory it would be good to have an option to skip the file and directory counts.

      Moreover, currently one can only check the quota if you have access to all the directories underneath. For example, if I have a large home directory under /user/joep and I host some files for another user in a sub-directory, the moment they create an unreadable sub-directory under my home I can no longer check what my quota is. Understood that I cannot check the current file counts unless I can iterate through all the usage, but for administrative purposes it is nice to be able to get the current quota setting on a directory without the need to iterate through and run into permission issues on sub-directories.

      1. HDFS-8898.patch
        33 kB
        Ming Ma
      2. HDFS-8898-2.patch
        88 kB
        Ming Ma
      3. HDFS-8898-3.patch
        91 kB
        Ming Ma
      4. HDFS-8898-4.patch
        94 kB
        Ming Ma
      5. HDFS-8898-5.patch
        95 kB
        Ming Ma
      6. HDFS-8898-6.patch
        95 kB
        Ming Ma
      7. HDFS-8898-branch-2.patch
        91 kB
        Ming Ma

        Issue Links

          Activity

          Hide
          jlowe Jason Lowe added a comment -

          This would solve a significant annoyance with computing quotas on a shared tree. However I think it has security implications. If one can get the quota totals for the entire tree then they can calculate what must be used by the parts they cannot access via quota_usage - usage_visible. If what is being stored in the restricted area is sensitive (e.g.: records related to financials) then knowing how many files or the size of the restricted data could leak sensitive information.

          Show
          jlowe Jason Lowe added a comment - This would solve a significant annoyance with computing quotas on a shared tree. However I think it has security implications. If one can get the quota totals for the entire tree then they can calculate what must be used by the parts they cannot access via quota_usage - usage_visible. If what is being stored in the restricted area is sensitive (e.g.: records related to financials) then knowing how many files or the size of the restricted data could leak sensitive information.
          Hide
          jrottinghuis Joep Rottinghuis added a comment -

          So it sounds like we're discussing two things here:
          1) Getting the quota itself for a directory that a user has access to. There seems to be little security concerns with this.
          2) Getting the quota, and the "ContentSummary" / count / usage for a directory that a user has access to, even if they might not have access to all the sub-directories. This is where Jason Lowe pointed out that there could be a potential security implication.

          Even with yielding the NN lock, it seems the NN can still lock for ~1 sec per 10M files in a sub-directory to check the entire sub-directory sub-directory tree for permissions.
          To address the potential security implications for 2) we could either make this a cluster-wide (final) config value, or we could do something with an extended attribute on the directory itself to allow or disallow a particular directory to be traversed (or not).

          1) would give a huge performance boost for the cases when people just want to know what the quota is.
          2) would give a huge performance boost for the cases when people want to know a quota plus what's left for large directories relatively high in the directory structure (let alone / on a huge namespace of many tens of millions of files).

          Show
          jrottinghuis Joep Rottinghuis added a comment - So it sounds like we're discussing two things here: 1) Getting the quota itself for a directory that a user has access to. There seems to be little security concerns with this. 2) Getting the quota, and the "ContentSummary" / count / usage for a directory that a user has access to, even if they might not have access to all the sub-directories. This is where Jason Lowe pointed out that there could be a potential security implication. Even with yielding the NN lock, it seems the NN can still lock for ~1 sec per 10M files in a sub-directory to check the entire sub-directory sub-directory tree for permissions. To address the potential security implications for 2) we could either make this a cluster-wide (final) config value, or we could do something with an extended attribute on the directory itself to allow or disallow a particular directory to be traversed (or not). 1) would give a huge performance boost for the cases when people just want to know what the quota is. 2) would give a huge performance boost for the cases when people want to know a quota plus what's left for large directories relatively high in the directory structure (let alone / on a huge namespace of many tens of millions of files).
          Hide
          mingma Ming Ma added a comment -

          Can we provide optimizations for specific use cases without sacrificing the overall security requirement Jason mentioned above?

          • Allow super users to get a simplified version of ContentSummary quickly. To add to the Joep’s scenarios, we have scenarios which require quota and high-level usage “quota + namespace files/dirs count + disk consumption”, which is a subset of what ContentSummary provides(file length, distinction between files and dirs aren’t important). If we provide a ContentSummaryV2 with only these fields, NN can just return usage data cached on directory objects with quota set. For regular user or directories without quota set, traversal is still required.
          • Support other users besides super users. The permissions check is already skipped if the caller is a super user. If we define something like power user which have less power than super user but have read access to all directories, we can apply the same optimization to these power users. At the minimal, proxy users that can represent everyone should be treated like super users for the getContentSummary scenario.
          Show
          mingma Ming Ma added a comment - Can we provide optimizations for specific use cases without sacrificing the overall security requirement Jason mentioned above? Allow super users to get a simplified version of ContentSummary quickly. To add to the Joep’s scenarios, we have scenarios which require quota and high-level usage “quota + namespace files/dirs count + disk consumption”, which is a subset of what ContentSummary provides(file length, distinction between files and dirs aren’t important). If we provide a ContentSummaryV2 with only these fields, NN can just return usage data cached on directory objects with quota set. For regular user or directories without quota set, traversal is still required. Support other users besides super users. The permissions check is already skipped if the caller is a super user. If we define something like power user which have less power than super user but have read access to all directories, we can apply the same optimization to these power users. At the minimal, proxy users that can represent everyone should be treated like super users for the getContentSummary scenario.
          Hide
          mingma Ming Ma added a comment -

          Here is the patch just to illustrate the idea. Applications have the option to get QuotaUsage for any directory that has quota set. It contains the quota and the usage. This allows NN to directly use the cached data.

          For a regular user, NN's recursive file permission check still takes time, but at least getting the actual usage is fast. So the overall latency of getting quota usage is faster than getContentSummary. For a super user, given there is no more traversal so it will just take few milliseconds for any large directory.

          Show
          mingma Ming Ma added a comment - Here is the patch just to illustrate the idea. Applications have the option to get QuotaUsage for any directory that has quota set. It contains the quota and the usage. This allows NN to directly use the cached data. For a regular user, NN's recursive file permission check still takes time, but at least getting the actual usage is fast. So the overall latency of getting quota usage is faster than getContentSummary. For a super user, given there is no more traversal so it will just take few milliseconds for any large directory.
          Hide
          kihwal Kihwal Lee added a comment -

          +1 to the approach.

          Show
          kihwal Kihwal Lee added a comment - +1 to the approach.
          Hide
          mingma Ming Ma added a comment -

          Thanks Kihwal Lee! I will update the patch with new unit tests.

          Show
          mingma Ming Ma added a comment - Thanks Kihwal Lee ! I will update the patch with new unit tests.
          Hide
          mingma Ming Ma added a comment -

          Here is the updated patch with new unit tests. Another change in the updated patch is to have ContentSummary reuse QuotaUsage structure given they have lots of overlaps. I have manually verified that existing applications compiled with the old ContentSummary will still work with the new binary without recompilation. Procobuf definition for these two structures remain separate.

          Show
          mingma Ming Ma added a comment - Here is the updated patch with new unit tests. Another change in the updated patch is to have ContentSummary reuse QuotaUsage structure given they have lots of overlaps. I have manually verified that existing applications compiled with the old ContentSummary will still work with the new binary without recompilation. Procobuf definition for these two structures remain separate.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Thanks Ming Ma for the latest patch.
          It looks really nice.

          Below are some minor commments
          1. DFSClient#getQuotaUsage(..) should call NN RPC inside Trace span.
          2. DFSClient#getQuotaUsage(..) should call checkOpen() before going for NN RPC call
          3. Option added to Count command needs document update.
          4. Following code too is expected to be inside fsd lock.

          src = fsd.resolvePath(pc, src, pathComponents);
              final INodesInPath iip = fsd.getINodesInPath(src, false);
              if (fsd.isPermissionEnabled()) {
                fsd.checkPermission(pc, iip, false, null, null, null,
                FsAction.READ_EXECUTE);
              }

          5. FSNamesystem#getQuotaUsage(..) needs formatting.

          Show
          vinayrpet Vinayakumar B added a comment - Thanks Ming Ma for the latest patch. It looks really nice. Below are some minor commments 1. DFSClient#getQuotaUsage(..) should call NN RPC inside Trace span. 2. DFSClient#getQuotaUsage(..) should call checkOpen() before going for NN RPC call 3. Option added to Count command needs document update. 4. Following code too is expected to be inside fsd lock. src = fsd.resolvePath(pc, src, pathComponents); final INodesInPath iip = fsd.getINodesInPath(src, false ); if (fsd.isPermissionEnabled()) { fsd.checkPermission(pc, iip, false , null , null , null , FsAction.READ_EXECUTE); } 5. FSNamesystem#getQuotaUsage(..) needs formatting.
          Hide
          vinayrpet Vinayakumar B added a comment -

          Seems like precommit didnt detect this.
          triggered one run now.

          Show
          vinayrpet Vinayakumar B added a comment - Seems like precommit didnt detect this. triggered one run now.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 5s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          +1 mvninstall 2m 50s trunk passed
          +1 compile 4m 21s trunk passed with JDK v1.8.0_60
          +1 compile 4m 12s trunk passed with JDK v1.7.0_79
          +1 checkstyle 0m 56s trunk passed
          +1 mvneclipse 0m 39s trunk passed
          -1 findbugs 1m 48s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
          +1 javadoc 2m 21s trunk passed with JDK v1.8.0_60
          +1 javadoc 3m 11s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 21s hadoop-hdfs in the patch failed.
          +1 compile 4m 19s the patch passed with JDK v1.8.0_60
          +1 cc 4m 19s the patch passed
          +1 javac 4m 19s the patch passed
          +1 compile 4m 8s the patch passed with JDK v1.7.0_79
          +1 cc 4m 8s the patch passed
          +1 javac 4m 8s the patch passed
          -1 checkstyle 0m 57s Patch generated 57 new checkstyle issues in root (total was 824, now 841).
          +1 mvneclipse 0m 39s the patch passed
          -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          -1 findbugs 1m 50s hadoop-common-project/hadoop-common introduced 2 new FindBugs issues.
          +1 javadoc 2m 18s the patch passed with JDK v1.8.0_60
          -1 javadoc 8m 8s hadoop-common-project_hadoop-common-jdk1.7.0_79 with JDK v1.7.0_79 has problems.
          +1 javadoc 3m 13s the patch passed with JDK v1.7.0_79
          -1 unit 6m 7s hadoop-common in the patch failed with JDK v1.8.0_60.
          -1 unit 51m 19s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_60.
          -1 unit 6m 27s hadoop-common in the patch failed with JDK v1.7.0_79.
          -1 unit 50m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79.
          -1 asflicense 0m 19s Patch generated 56 ASF License warnings.
          165m 7s



          Reason Tests
          FindBugs module:hadoop-common-project/hadoop-common
            org.apache.hadoop.fs.ContentSummary doesn't override QuotaUsage.equals(Object) At ContentSummary.java:At ContentSummary.java:[line 1]
            org.apache.hadoop.fs.QuotaUsage defines equals and uses Object.hashCode() At QuotaUsage.java:Object.hashCode() At QuotaUsage.java:[line 183]
          JDK v1.8.0_60 Failed junit tests hadoop.metrics2.sink.TestFileSink
            hadoop.security.TestShellBasedIdMapping
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.blockmanagement.TestNodeCount
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
          JDK v1.7.0_79 Failed junit tests hadoop.metrics2.sink.TestFileSink
            hadoop.hdfs.TestAclsEndToEnd



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770122/HDFS-8898-2.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile xml cc
          uname Linux 7d8bf7723bc9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh
          git revision trunk / 957f031
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/whitespace-eol.txt
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html
          javadoc hadoop-common-project_hadoop-common-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_79-diff.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13358/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: .
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13358/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 5s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. +1 mvninstall 2m 50s trunk passed +1 compile 4m 21s trunk passed with JDK v1.8.0_60 +1 compile 4m 12s trunk passed with JDK v1.7.0_79 +1 checkstyle 0m 56s trunk passed +1 mvneclipse 0m 39s trunk passed -1 findbugs 1m 48s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 2m 21s trunk passed with JDK v1.8.0_60 +1 javadoc 3m 11s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 21s hadoop-hdfs in the patch failed. +1 compile 4m 19s the patch passed with JDK v1.8.0_60 +1 cc 4m 19s the patch passed +1 javac 4m 19s the patch passed +1 compile 4m 8s the patch passed with JDK v1.7.0_79 +1 cc 4m 8s the patch passed +1 javac 4m 8s the patch passed -1 checkstyle 0m 57s Patch generated 57 new checkstyle issues in root (total was 824, now 841). +1 mvneclipse 0m 39s the patch passed -1 whitespace 0m 0s The patch has 2 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. -1 findbugs 1m 50s hadoop-common-project/hadoop-common introduced 2 new FindBugs issues. +1 javadoc 2m 18s the patch passed with JDK v1.8.0_60 -1 javadoc 8m 8s hadoop-common-project_hadoop-common-jdk1.7.0_79 with JDK v1.7.0_79 has problems. +1 javadoc 3m 13s the patch passed with JDK v1.7.0_79 -1 unit 6m 7s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 51m 19s hadoop-hdfs in the patch failed with JDK v1.8.0_60. +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_60. -1 unit 6m 27s hadoop-common in the patch failed with JDK v1.7.0_79. -1 unit 50m 11s hadoop-hdfs in the patch failed with JDK v1.7.0_79. +1 unit 0m 53s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79. -1 asflicense 0m 19s Patch generated 56 ASF License warnings. 165m 7s Reason Tests FindBugs module:hadoop-common-project/hadoop-common   org.apache.hadoop.fs.ContentSummary doesn't override QuotaUsage.equals(Object) At ContentSummary.java:At ContentSummary.java: [line 1]   org.apache.hadoop.fs.QuotaUsage defines equals and uses Object.hashCode() At QuotaUsage.java:Object.hashCode() At QuotaUsage.java: [line 183] JDK v1.8.0_60 Failed junit tests hadoop.metrics2.sink.TestFileSink   hadoop.security.TestShellBasedIdMapping   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.blockmanagement.TestNodeCount   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes JDK v1.7.0_79 Failed junit tests hadoop.metrics2.sink.TestFileSink   hadoop.hdfs.TestAclsEndToEnd Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-03 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770122/HDFS-8898-2.patch JIRA Issue HDFS-8898 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile xml cc uname Linux 7d8bf7723bc9 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-1a9afee/precommit/personality/hadoop.sh git revision trunk / 957f031 Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/whitespace-eol.txt findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/new-findbugs-hadoop-common-project_hadoop-common.html javadoc hadoop-common-project_hadoop-common-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_79-diff.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13358/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13358/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13358/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          Thanks Vinayakumar B! Here is the updated patch that addresses your comments, findbugs and many of the checkstyle issues.

          3. Option added to Count command needs document update.

          It looks like the document also misses the storage type quota. I updated it as part of this change. If more work is needed for storage type quota, I can open a new jira.

          4. Following code too is expected to be inside fsd lock.

          Fixed. good point. It seems getFileInfo and getContentSummary have similar issues. Any ideas? If we need to fix those cases, maybe it is better to open a new jira for that.

          Show
          mingma Ming Ma added a comment - Thanks Vinayakumar B ! Here is the updated patch that addresses your comments, findbugs and many of the checkstyle issues. 3. Option added to Count command needs document update. It looks like the document also misses the storage type quota. I updated it as part of this change. If more work is needed for storage type quota, I can open a new jira. 4. Following code too is expected to be inside fsd lock. Fixed. good point. It seems getFileInfo and getContentSummary have similar issues. Any ideas? If we need to fix those cases, maybe it is better to open a new jira for that.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          +1 mvninstall 3m 18s trunk passed
          +1 compile 5m 13s trunk passed with JDK v1.8.0_60
          +1 compile 4m 57s trunk passed with JDK v1.7.0_79
          +1 checkstyle 1m 14s trunk passed
          +1 mvnsite 2m 37s trunk passed
          +1 mvneclipse 0m 43s trunk passed
          -1 findbugs 2m 22s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs
          +1 javadoc 3m 30s trunk passed with JDK v1.8.0_60
          +1 javadoc 4m 14s trunk passed with JDK v1.7.0_79
          -1 mvninstall 0m 24s hadoop-hdfs in the patch failed.
          +1 compile 5m 18s the patch passed with JDK v1.8.0_60
          +1 cc 5m 18s the patch passed
          +1 javac 5m 18s the patch passed
          +1 compile 4m 55s the patch passed with JDK v1.7.0_79
          +1 cc 4m 55s the patch passed
          +1 javac 4m 55s the patch passed
          -1 checkstyle 1m 10s Patch generated 26 new checkstyle issues in root (total was 824, now 810).
          +1 mvnsite 2m 32s the patch passed
          +1 mvneclipse 0m 46s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 7m 10s the patch passed
          +1 javadoc 2m 48s the patch passed with JDK v1.8.0_60
          -1 javadoc 9m 55s hadoop-common-project_hadoop-common-jdk1.7.0_79 with JDK v1.7.0_79 generated 1 new issues (was 13, now 14).
          +1 javadoc 3m 54s the patch passed with JDK v1.7.0_79
          -1 unit 7m 32s hadoop-common in the patch failed with JDK v1.8.0_60.
          -1 unit 59m 23s hadoop-hdfs in the patch failed with JDK v1.8.0_60.
          +1 unit 0m 54s hadoop-hdfs-client in the patch passed with JDK v1.8.0_60.
          -1 unit 7m 19s hadoop-common in the patch failed with JDK v1.7.0_79.
          -1 unit 55m 8s hadoop-hdfs in the patch failed with JDK v1.7.0_79.
          +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79.
          -1 asflicense 0m 20s Patch generated 58 ASF License warnings.
          196m 55s



          Reason Tests
          JDK v1.8.0_60 Failed junit tests hadoop.fs.shell.TestCount
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020
            hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160
          JDK v1.7.0_79 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag
            hadoop.metrics2.impl.TestMetricsSystemImpl
            hadoop.fs.shell.TestCount
            hadoop.fs.TestSymlinkLocalFSFileContext
            hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.ipc.TestDecayRpcScheduler
            hadoop.hdfs.shortcircuit.TestShortCircuitCache
            hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes



          Subsystem Report/Notes
          Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770526/HDFS-8898-3.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile site mvnsite xml cc
          uname Linux d0aa6d1ace72 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh
          git revision trunk / 194251c
          Default Java 1.7.0_79
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/diff-checkstyle-root.txt
          javadoc hadoop-common-project_hadoop-common-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt
          JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13376/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: .
          Max memory used 226MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13376/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. +1 mvninstall 3m 18s trunk passed +1 compile 5m 13s trunk passed with JDK v1.8.0_60 +1 compile 4m 57s trunk passed with JDK v1.7.0_79 +1 checkstyle 1m 14s trunk passed +1 mvnsite 2m 37s trunk passed +1 mvneclipse 0m 43s trunk passed -1 findbugs 2m 22s hadoop-hdfs-project/hadoop-hdfs in trunk cannot run convertXmlToText from findbugs +1 javadoc 3m 30s trunk passed with JDK v1.8.0_60 +1 javadoc 4m 14s trunk passed with JDK v1.7.0_79 -1 mvninstall 0m 24s hadoop-hdfs in the patch failed. +1 compile 5m 18s the patch passed with JDK v1.8.0_60 +1 cc 5m 18s the patch passed +1 javac 5m 18s the patch passed +1 compile 4m 55s the patch passed with JDK v1.7.0_79 +1 cc 4m 55s the patch passed +1 javac 4m 55s the patch passed -1 checkstyle 1m 10s Patch generated 26 new checkstyle issues in root (total was 824, now 810). +1 mvnsite 2m 32s the patch passed +1 mvneclipse 0m 46s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 7m 10s the patch passed +1 javadoc 2m 48s the patch passed with JDK v1.8.0_60 -1 javadoc 9m 55s hadoop-common-project_hadoop-common-jdk1.7.0_79 with JDK v1.7.0_79 generated 1 new issues (was 13, now 14). +1 javadoc 3m 54s the patch passed with JDK v1.7.0_79 -1 unit 7m 32s hadoop-common in the patch failed with JDK v1.8.0_60. -1 unit 59m 23s hadoop-hdfs in the patch failed with JDK v1.8.0_60. +1 unit 0m 54s hadoop-hdfs-client in the patch passed with JDK v1.8.0_60. -1 unit 7m 19s hadoop-common in the patch failed with JDK v1.7.0_79. -1 unit 55m 8s hadoop-hdfs in the patch failed with JDK v1.7.0_79. +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_79. -1 asflicense 0m 20s Patch generated 58 ASF License warnings. 196m 55s Reason Tests JDK v1.8.0_60 Failed junit tests hadoop.fs.shell.TestCount   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure020   hadoop.hdfs.TestDFSStripedOutputStreamWithFailure160 JDK v1.7.0_79 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag   hadoop.metrics2.impl.TestMetricsSystemImpl   hadoop.fs.shell.TestCount   hadoop.fs.TestSymlinkLocalFSFileContext   hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.ipc.TestDecayRpcScheduler   hadoop.hdfs.shortcircuit.TestShortCircuitCache   hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes Subsystem Report/Notes Docker Client=1.7.1 Server=1.7.1 Image:test-patch-base-hadoop-date2015-11-04 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12770526/HDFS-8898-3.patch JIRA Issue HDFS-8898 Optional Tests asflicense javac javadoc mvninstall unit findbugs checkstyle compile site mvnsite xml cc uname Linux d0aa6d1ace72 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-d0f6847/precommit/personality/hadoop.sh git revision trunk / 194251c Default Java 1.7.0_79 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_60 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_79 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs.html mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/diff-checkstyle-root.txt javadoc hadoop-common-project_hadoop-common-jdk1.7.0_79: https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/diff-javadoc-javadoc-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_60.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_79.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_79.txt JDK v1.7.0_79 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13376/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13376/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . Max memory used 226MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13376/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          New patch to address TestCount and the similar issue found in HDFS-9413.

          Show
          mingma Ming Ma added a comment - New patch to address TestCount and the similar issue found in HDFS-9413 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 6s docker + precommit patch detected.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
          +1 mvninstall 8m 56s trunk passed
          +1 compile 9m 37s trunk passed with JDK v1.8.0_66
          +1 compile 9m 47s trunk passed with JDK v1.7.0_85
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 2m 52s trunk passed
          +1 mvneclipse 0m 47s trunk passed
          +1 findbugs 6m 33s trunk passed
          +1 javadoc 2m 43s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 39s trunk passed with JDK v1.7.0_85
          -1 mvninstall 0m 26s hadoop-hdfs in the patch failed.
          +1 compile 9m 38s the patch passed with JDK v1.8.0_66
          +1 cc 9m 38s the patch passed
          -1 javac 20m 45s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 14 new issues (was 779, now 779).
          +1 javac 9m 38s the patch passed
          +1 compile 10m 16s the patch passed with JDK v1.7.0_85
          +1 cc 10m 16s the patch passed
          -1 javac 31m 1s root-jdk1.7.0_85 with JDK v1.7.0_85 generated 13 new issues (was 772, now 772).
          +1 javac 10m 16s the patch passed
          -1 checkstyle 1m 7s Patch generated 24 new checkstyle issues in root (total was 823, now 807).
          +1 mvnsite 2m 52s the patch passed
          +1 mvneclipse 0m 45s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 6m 35s the patch passed
          +1 javadoc 2m 39s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 32s the patch passed with JDK v1.7.0_85
          -1 unit 7m 52s hadoop-common in the patch failed with JDK v1.8.0_66.
          -1 unit 63m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 31s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          +1 unit 9m 12s hadoop-common in the patch passed with JDK v1.7.0_85.
          -1 unit 59m 17s hadoop-hdfs in the patch failed with JDK v1.7.0_85.
          +1 unit 1m 4s hadoop-hdfs-client in the patch passed with JDK v1.7.0_85.
          -1 asflicense 0m 22s Patch generated 58 ASF License warnings.
          230m 9s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.ipc.TestDecayRpcScheduler
            hadoop.security.ssl.TestReloadingX509TrustManager
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.namenode.TestNamenodeCapacityReport
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots
            hadoop.hdfs.TestHDFSFileSystemContract
          JDK v1.7.0_85 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.TestRollingUpgrade
            hadoop.hdfs.server.datanode.TestBlockScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:date2015-11-19
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773111/HDFS-8898-4.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux c944f9ca6306 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh
          git revision trunk / 23a130a
          findbugs v3.0.0
          mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt
          javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt
          javac root-jdk1.7.0_85: https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_85.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt
          JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13553/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: .
          Max memory used 78MB
          Powered by Apache Yetus http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13553/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 6s docker + precommit patch detected. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. +1 mvninstall 8m 56s trunk passed +1 compile 9m 37s trunk passed with JDK v1.8.0_66 +1 compile 9m 47s trunk passed with JDK v1.7.0_85 +1 checkstyle 1m 6s trunk passed +1 mvnsite 2m 52s trunk passed +1 mvneclipse 0m 47s trunk passed +1 findbugs 6m 33s trunk passed +1 javadoc 2m 43s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 39s trunk passed with JDK v1.7.0_85 -1 mvninstall 0m 26s hadoop-hdfs in the patch failed. +1 compile 9m 38s the patch passed with JDK v1.8.0_66 +1 cc 9m 38s the patch passed -1 javac 20m 45s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 14 new issues (was 779, now 779). +1 javac 9m 38s the patch passed +1 compile 10m 16s the patch passed with JDK v1.7.0_85 +1 cc 10m 16s the patch passed -1 javac 31m 1s root-jdk1.7.0_85 with JDK v1.7.0_85 generated 13 new issues (was 772, now 772). +1 javac 10m 16s the patch passed -1 checkstyle 1m 7s Patch generated 24 new checkstyle issues in root (total was 823, now 807). +1 mvnsite 2m 52s the patch passed +1 mvneclipse 0m 45s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 6m 35s the patch passed +1 javadoc 2m 39s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 32s the patch passed with JDK v1.7.0_85 -1 unit 7m 52s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 63m 1s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 1m 31s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. +1 unit 9m 12s hadoop-common in the patch passed with JDK v1.7.0_85. -1 unit 59m 17s hadoop-hdfs in the patch failed with JDK v1.7.0_85. +1 unit 1m 4s hadoop-hdfs-client in the patch passed with JDK v1.7.0_85. -1 asflicense 0m 22s Patch generated 58 ASF License warnings. 230m 9s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.ipc.TestDecayRpcScheduler   hadoop.security.ssl.TestReloadingX509TrustManager   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.server.namenode.ha.TestSeveralNameNodes   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.namenode.TestNamenodeCapacityReport   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots   hadoop.hdfs.TestHDFSFileSystemContract JDK v1.7.0_85 Failed junit tests hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestRollingUpgrade   hadoop.hdfs.server.datanode.TestBlockScanner Subsystem Report/Notes Docker Image:yetus/hadoop:date2015-11-19 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12773111/HDFS-8898-4.patch JIRA Issue HDFS-8898 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux c944f9ca6306 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HDFS-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh git revision trunk / 23a130a findbugs v3.0.0 mvninstall https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-mvninstall-hadoop-hdfs-project_hadoop-hdfs.txt javac root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-compile-javac-root-jdk1.8.0_66.txt javac root-jdk1.7.0_85: https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-compile-javac-root-jdk1.7.0_85.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_85.txt JDK v1.7.0_85 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/13553/testReport/ asflicense https://builds.apache.org/job/PreCommit-HDFS-Build/13553/artifact/patchprocess/patch-asflicense-problems.txt modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project/hadoop-hdfs-client U: . Max memory used 78MB Powered by Apache Yetus http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/13553/console This message was automatically generated.
          Hide
          mingma Ming Ma added a comment -

          mvninstall failure is due to the known issue, where another job on the same machine updated mvn cache for hadoop-hdfs-project/hadoop-hdfs-client and thus caused the build of hadoop-hdfs-project/hadoop-hdfs to fail. The failed unit tests aren't related. Vinayakumar B, Kihwal Lee, any more comments?

          Show
          mingma Ming Ma added a comment - mvninstall failure is due to the known issue, where another job on the same machine updated mvn cache for hadoop-hdfs-project/hadoop-hdfs-client and thus caused the build of hadoop-hdfs-project/hadoop-hdfs to fail. The failed unit tests aren't related. Vinayakumar B , Kihwal Lee , any more comments?
          Hide
          kihwal Kihwal Lee added a comment -

          How does a new client work with an older namenode that does not implement getQuotaUsage?
          If we are not planning on making it work, we should probably show more user friendly error message.
          Other than that the patch looks good. Also it still applies!

          Show
          kihwal Kihwal Lee added a comment - How does a new client work with an older namenode that does not implement getQuotaUsage? If we are not planning on making it work, we should probably show more user friendly error message. Other than that the patch looks good. Also it still applies!
          Hide
          mingma Ming Ma added a comment -

          Thanks Kihwal Lee!

          In the case of new client talking to older namenode, we could have DFSClient fall back to use the existing getContentSummary RPC. Here is the updated patch. I have tested this specific case manually.

          Show
          mingma Ming Ma added a comment - Thanks Kihwal Lee ! In the case of new client talking to older namenode, we could have DFSClient fall back to use the existing getContentSummary RPC. Here is the updated patch. I have tested this specific case manually.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 7m 41s trunk passed
          +1 compile 6m 28s trunk passed with JDK v1.8.0_66
          +1 compile 6m 58s trunk passed with JDK v1.7.0_91
          +1 checkstyle 1m 11s trunk passed
          +1 mvnsite 2m 29s trunk passed
          +1 mvneclipse 0m 39s trunk passed
          +1 findbugs 5m 34s trunk passed
          +1 javadoc 2m 18s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 15s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 24s Maven dependency ordering for patch
          +1 mvninstall 2m 51s the patch passed
          +1 compile 6m 17s the patch passed with JDK v1.8.0_66
          -1 cc 9m 32s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new + 9 unchanged - 1 fixed = 10 total (was 10)
          +1 cc 6m 17s the patch passed
          +1 javac 6m 17s the patch passed
          +1 compile 6m 50s the patch passed with JDK v1.7.0_91
          +1 cc 6m 50s the patch passed
          +1 javac 6m 50s the patch passed
          -1 checkstyle 1m 6s root: patch generated 20 new + 702 unchanged - 36 fixed = 722 total (was 738)
          +1 mvnsite 2m 24s the patch passed
          +1 mvneclipse 0m 40s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 6m 13s the patch passed
          +1 javadoc 2m 14s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 10s the patch passed with JDK v1.7.0_91
          +1 unit 6m 21s hadoop-common in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 50m 57s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 6m 44s hadoop-common in the patch passed with JDK v1.7.0_91.
          +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          +1 unit 49m 42s hadoop-hdfs in the patch passed with JDK v1.7.0_91.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          186m 57s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783230/HDFS-8898-5.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux 1b8258675f0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 1acc509
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          cc root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_66.txt
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14167/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14167/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 7m 41s trunk passed +1 compile 6m 28s trunk passed with JDK v1.8.0_66 +1 compile 6m 58s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 11s trunk passed +1 mvnsite 2m 29s trunk passed +1 mvneclipse 0m 39s trunk passed +1 findbugs 5m 34s trunk passed +1 javadoc 2m 18s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 15s trunk passed with JDK v1.7.0_91 0 mvndep 0m 24s Maven dependency ordering for patch +1 mvninstall 2m 51s the patch passed +1 compile 6m 17s the patch passed with JDK v1.8.0_66 -1 cc 9m 32s root-jdk1.8.0_66 with JDK v1.8.0_66 generated 1 new + 9 unchanged - 1 fixed = 10 total (was 10) +1 cc 6m 17s the patch passed +1 javac 6m 17s the patch passed +1 compile 6m 50s the patch passed with JDK v1.7.0_91 +1 cc 6m 50s the patch passed +1 javac 6m 50s the patch passed -1 checkstyle 1m 6s root: patch generated 20 new + 702 unchanged - 36 fixed = 722 total (was 738) +1 mvnsite 2m 24s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 6m 13s the patch passed +1 javadoc 2m 14s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 10s the patch passed with JDK v1.7.0_91 +1 unit 6m 21s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 0m 51s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 50m 57s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 6m 44s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 0m 58s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. +1 unit 49m 42s hadoop-hdfs in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 186m 57s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783230/HDFS-8898-5.patch JIRA Issue HDFS-8898 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux 1b8258675f0c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 1acc509 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 cc root-jdk1.8.0_66: https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/diff-compile-cc-root-jdk1.8.0_66.txt checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14167/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14167/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14167/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          The change looks good, but it would be better if the log level was debug or info at most. Please address the checkstyle warnings as much as possible.

          Show
          kihwal Kihwal Lee added a comment - The change looks good, but it would be better if the log level was debug or info at most. Please address the checkstyle warnings as much as possible.
          Hide
          mingma Ming Ma added a comment -

          Thanks Kihwal Lee. Here is the patch with log level and some checkstyle fixes.

          Show
          mingma Ming Ma added a comment - Thanks Kihwal Lee . Here is the patch with log level and some checkstyle fixes.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 9 new or modified test files.
          0 mvndep 0m 24s Maven dependency ordering for branch
          +1 mvninstall 7m 31s trunk passed
          +1 compile 5m 51s trunk passed with JDK v1.8.0_66
          +1 compile 6m 46s trunk passed with JDK v1.7.0_91
          +1 checkstyle 1m 6s trunk passed
          +1 mvnsite 2m 27s trunk passed
          +1 mvneclipse 0m 40s trunk passed
          +1 findbugs 5m 28s trunk passed
          +1 javadoc 2m 30s trunk passed with JDK v1.8.0_66
          +1 javadoc 3m 42s trunk passed with JDK v1.7.0_91
          0 mvndep 0m 24s Maven dependency ordering for patch
          +1 mvninstall 2m 57s the patch passed
          +1 compile 6m 15s the patch passed with JDK v1.8.0_66
          +1 cc 6m 15s the patch passed
          +1 javac 6m 15s the patch passed
          +1 compile 7m 8s the patch passed with JDK v1.7.0_91
          +1 cc 7m 8s the patch passed
          +1 javac 7m 8s the patch passed
          -1 checkstyle 1m 5s root: patch generated 15 new + 702 unchanged - 36 fixed = 717 total (was 738)
          +1 mvnsite 2m 26s the patch passed
          +1 mvneclipse 0m 40s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 6m 18s the patch passed
          +1 javadoc 2m 20s the patch passed with JDK v1.8.0_66
          +1 javadoc 3m 38s the patch passed with JDK v1.7.0_91
          +1 unit 6m 33s hadoop-common in the patch passed with JDK v1.8.0_66.
          +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 52m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 7m 23s hadoop-common in the patch passed with JDK v1.7.0_91.
          -1 unit 0m 25s hadoop-hdfs-client in the patch failed with JDK v1.7.0_91.
          -1 unit 0m 28s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 20s Patch does not generate ASF License warnings.
          140m 18s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestFSImageWithAcl
            hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783433/HDFS-8898-6.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux 31e52a0e851b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 7905788
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/diff-checkstyle-root.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client-jdk1.7.0_91.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14178/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 77MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14178/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 9 new or modified test files. 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 7m 31s trunk passed +1 compile 5m 51s trunk passed with JDK v1.8.0_66 +1 compile 6m 46s trunk passed with JDK v1.7.0_91 +1 checkstyle 1m 6s trunk passed +1 mvnsite 2m 27s trunk passed +1 mvneclipse 0m 40s trunk passed +1 findbugs 5m 28s trunk passed +1 javadoc 2m 30s trunk passed with JDK v1.8.0_66 +1 javadoc 3m 42s trunk passed with JDK v1.7.0_91 0 mvndep 0m 24s Maven dependency ordering for patch +1 mvninstall 2m 57s the patch passed +1 compile 6m 15s the patch passed with JDK v1.8.0_66 +1 cc 6m 15s the patch passed +1 javac 6m 15s the patch passed +1 compile 7m 8s the patch passed with JDK v1.7.0_91 +1 cc 7m 8s the patch passed +1 javac 7m 8s the patch passed -1 checkstyle 1m 5s root: patch generated 15 new + 702 unchanged - 36 fixed = 717 total (was 738) +1 mvnsite 2m 26s the patch passed +1 mvneclipse 0m 40s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 6m 18s the patch passed +1 javadoc 2m 20s the patch passed with JDK v1.8.0_66 +1 javadoc 3m 38s the patch passed with JDK v1.7.0_91 +1 unit 6m 33s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 0m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 52m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 7m 23s hadoop-common in the patch passed with JDK v1.7.0_91. -1 unit 0m 25s hadoop-hdfs-client in the patch failed with JDK v1.7.0_91. -1 unit 0m 28s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 20s Patch does not generate ASF License warnings. 140m 18s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.hdfs.server.namenode.TestFSImageWithAcl   hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783433/HDFS-8898-6.patch JIRA Issue HDFS-8898 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux 31e52a0e851b 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 7905788 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/diff-checkstyle-root.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-client-jdk1.7.0_91.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14178/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14178/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 77MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14178/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          The patch looks good.
          The checkstyle warnings seem to be due to the builder methods being non-void. I think we can ignore that.
          Please make sure the unit test failures are unrelated and whether there already is a jira that reported the issue.

          Show
          kihwal Kihwal Lee added a comment - The patch looks good. The checkstyle warnings seem to be due to the builder methods being non-void. I think we can ignore that. Please make sure the unit test failures are unrelated and whether there already is a jira that reported the issue.
          Hide
          mingma Ming Ma added a comment -

          Ran these local tests locally with java 1.7 and 1.8. They all passed. No jiras have been opened so far. Let me open a jira for TestNNThroughputBenchmark. It has had issues as reported in other jenkins jobs.

          Show
          mingma Ming Ma added a comment - Ran these local tests locally with java 1.7 and 1.8. They all passed. No jiras have been opened so far. Let me open a jira for TestNNThroughputBenchmark. It has had issues as reported in other jenkins jobs.
          Hide
          kihwal Kihwal Lee added a comment -

          +1

          Show
          kihwal Kihwal Lee added a comment - +1
          Hide
          kihwal Kihwal Lee added a comment -

          It looks like branch-2 & branch-2.8 needs a different patch.

          Show
          kihwal Kihwal Lee added a comment - It looks like branch-2 & branch-2.8 needs a different patch.
          Hide
          mingma Ming Ma added a comment -

          Here is the patch for branch-2. I have run it with all unit tests. The patch also applies to branch-2.8.

          Show
          mingma Ming Ma added a comment - Here is the patch for branch-2. I have run it with all unit tests. The patch also applies to branch-2.8.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 8 new or modified test files.
          0 mvndep 2m 18s Maven dependency ordering for branch
          +1 mvninstall 9m 57s branch-2 passed
          +1 compile 13m 39s branch-2 passed with JDK v1.8.0_66
          +1 compile 11m 19s branch-2 passed with JDK v1.7.0_91
          +1 checkstyle 1m 46s branch-2 passed
          +1 mvnsite 3m 46s branch-2 passed
          +1 mvneclipse 1m 7s branch-2 passed
          -1 findbugs 3m 7s hadoop-common-project/hadoop-common in branch-2 has 5 extant Findbugs warnings.
          -1 findbugs 2m 43s hadoop-hdfs-project/hadoop-hdfs-client in branch-2 has 5 extant Findbugs warnings.
          +1 javadoc 4m 3s branch-2 passed with JDK v1.8.0_66
          +1 javadoc 4m 35s branch-2 passed with JDK v1.7.0_91
          0 mvndep 0m 33s Maven dependency ordering for patch
          +1 mvninstall 2m 51s the patch passed
          +1 compile 14m 0s the patch passed with JDK v1.8.0_66
          +1 cc 14m 0s the patch passed
          +1 javac 14m 0s the patch passed
          +1 compile 11m 13s the patch passed with JDK v1.7.0_91
          +1 cc 11m 13s the patch passed
          +1 javac 11m 13s the patch passed
          -1 checkstyle 1m 41s root: patch generated 16 new + 706 unchanged - 37 fixed = 722 total (was 743)
          +1 mvnsite 3m 48s the patch passed
          +1 mvneclipse 1m 1s the patch passed
          -1 whitespace 0m 0s The patch has 61 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 9m 14s the patch passed
          +1 javadoc 3m 37s the patch passed with JDK v1.8.0_66
          +1 javadoc 5m 4s the patch passed with JDK v1.7.0_91
          -1 unit 13m 22s hadoop-common in the patch failed with JDK v1.8.0_66.
          +1 unit 1m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66.
          -1 unit 78m 14s hadoop-hdfs in the patch failed with JDK v1.8.0_66.
          +1 unit 9m 59s hadoop-common in the patch passed with JDK v1.7.0_91.
          +1 unit 1m 20s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91.
          -1 unit 64m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_91.
          +1 asflicense 0m 35s Patch does not generate ASF License warnings.
          286m 39s



          Reason Tests
          JDK v1.8.0_66 Failed junit tests hadoop.io.compress.TestCodecPool
            hadoop.ipc.TestRPCWaitForProxy
            hadoop.fs.TestSymlinkLocalFSFileSystem
            hadoop.test.TestTimedOutTestsListener
            hadoop.fs.shell.find.TestPrint
            hadoop.fs.shell.find.TestPrint0
            hadoop.hdfs.TestDFSUpgradeFromImage
            hadoop.hdfs.TestDistributedFileSystem
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.mover.TestStorageMover
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.ha.TestHAAppend
            hadoop.fs.TestSymlinkHdfsFileContext
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
          JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner
            hadoop.hdfs.server.datanode.TestDirectoryScanner
            hadoop.hdfs.server.namenode.TestNNThroughputBenchmark



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:babe025
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783734/HDFS-8898-branch-2.patch
          JIRA Issue HDFS-8898
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc
          uname Linux 16404b484d0a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision branch-2 / 4d9efaf
          Default Java 1.7.0_91
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/diff-checkstyle-root.txt
          whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt
          JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14197/testReport/
          modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: .
          Max memory used 73MB
          Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14197/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 8 new or modified test files. 0 mvndep 2m 18s Maven dependency ordering for branch +1 mvninstall 9m 57s branch-2 passed +1 compile 13m 39s branch-2 passed with JDK v1.8.0_66 +1 compile 11m 19s branch-2 passed with JDK v1.7.0_91 +1 checkstyle 1m 46s branch-2 passed +1 mvnsite 3m 46s branch-2 passed +1 mvneclipse 1m 7s branch-2 passed -1 findbugs 3m 7s hadoop-common-project/hadoop-common in branch-2 has 5 extant Findbugs warnings. -1 findbugs 2m 43s hadoop-hdfs-project/hadoop-hdfs-client in branch-2 has 5 extant Findbugs warnings. +1 javadoc 4m 3s branch-2 passed with JDK v1.8.0_66 +1 javadoc 4m 35s branch-2 passed with JDK v1.7.0_91 0 mvndep 0m 33s Maven dependency ordering for patch +1 mvninstall 2m 51s the patch passed +1 compile 14m 0s the patch passed with JDK v1.8.0_66 +1 cc 14m 0s the patch passed +1 javac 14m 0s the patch passed +1 compile 11m 13s the patch passed with JDK v1.7.0_91 +1 cc 11m 13s the patch passed +1 javac 11m 13s the patch passed -1 checkstyle 1m 41s root: patch generated 16 new + 706 unchanged - 37 fixed = 722 total (was 743) +1 mvnsite 3m 48s the patch passed +1 mvneclipse 1m 1s the patch passed -1 whitespace 0m 0s The patch has 61 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 9m 14s the patch passed +1 javadoc 3m 37s the patch passed with JDK v1.8.0_66 +1 javadoc 5m 4s the patch passed with JDK v1.7.0_91 -1 unit 13m 22s hadoop-common in the patch failed with JDK v1.8.0_66. +1 unit 1m 50s hadoop-hdfs-client in the patch passed with JDK v1.8.0_66. -1 unit 78m 14s hadoop-hdfs in the patch failed with JDK v1.8.0_66. +1 unit 9m 59s hadoop-common in the patch passed with JDK v1.7.0_91. +1 unit 1m 20s hadoop-hdfs-client in the patch passed with JDK v1.7.0_91. -1 unit 64m 39s hadoop-hdfs in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 35s Patch does not generate ASF License warnings. 286m 39s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.io.compress.TestCodecPool   hadoop.ipc.TestRPCWaitForProxy   hadoop.fs.TestSymlinkLocalFSFileSystem   hadoop.test.TestTimedOutTestsListener   hadoop.fs.shell.find.TestPrint   hadoop.fs.shell.find.TestPrint0   hadoop.hdfs.TestDFSUpgradeFromImage   hadoop.hdfs.TestDistributedFileSystem   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.mover.TestStorageMover   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.ha.TestHAAppend   hadoop.fs.TestSymlinkHdfsFileContext   hadoop.hdfs.security.TestDelegationTokenForProxyUser JDK v1.7.0_91 Failed junit tests hadoop.hdfs.server.datanode.TestBlockScanner   hadoop.hdfs.server.datanode.TestDirectoryScanner   hadoop.hdfs.server.namenode.TestNNThroughputBenchmark Subsystem Report/Notes Docker Image:yetus/hadoop:babe025 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12783734/HDFS-8898-branch-2.patch JIRA Issue HDFS-8898 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle xml cc uname Linux 16404b484d0a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision branch-2 / 4d9efaf Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html findbugs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/branch-findbugs-hadoop-hdfs-project_hadoop-hdfs-client-warnings.html checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/diff-checkstyle-root.txt whitespace https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14197/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14197/testReport/ modules C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: . Max memory used 73MB Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14197/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment - - edited

          The patch looks to be a correct port of the trunk patch.

          Show
          kihwal Kihwal Lee added a comment - - edited The patch looks to be a correct port of the trunk patch.
          Hide
          kihwal Kihwal Lee added a comment -

          I've committed this to trunk, branch-2 and branch-2.8. Thanks for working on this Ming, and Vinay for providing the initial review.

          Show
          kihwal Kihwal Lee added a comment - I've committed this to trunk, branch-2 and branch-2.8. Thanks for working on this Ming, and Vinay for providing the initial review.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9167 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9167/)
          HDFS-8898. Create API and command-line argument to get quota and quota (kihwal: rev 2fd19b9674420e025af54a5bed12eb96478f8c48)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • hadoop-common-project/hadoop-common/src/test/resources/testConf.xml
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestQuotasWithHA.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestQuotaUsage.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsDefaultValue.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Count.java
          • hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestCount.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/cli/CLITestHelper.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ContentSummary.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java
          • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/QuotaUsage.java
          • hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9167 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9167/ ) HDFS-8898 . Create API and command-line argument to get quota and quota (kihwal: rev 2fd19b9674420e025af54a5bed12eb96478f8c48) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java hadoop-common-project/hadoop-common/src/test/resources/testConf.xml hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestQuotasWithHA.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestQuotaUsage.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSDirStatAndListingOp.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolTranslatorPB.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestFilterFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FileSystem.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/hdfs.proto hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ChRootedFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/fs/viewfs/TestViewFsDefaultValue.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocolPB/ClientNamenodeProtocolServerSideTranslatorPB.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSClient.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/Count.java hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/ClientProtocol.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestQuota.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/shell/TestCount.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/cli/CLITestHelper.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ContentSummary.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DistributedFileSystem.java hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/QuotaUsage.java hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestHarFileSystem.java
          Hide
          aw Allen Wittenauer added a comment -

          Should this have deprecated the public API created by HDFS-3000?

          Show
          aw Allen Wittenauer added a comment - Should this have deprecated the public API created by HDFS-3000 ?
          Hide
          mingma Ming Ma added a comment -

          Allen Wittenauer, HDFS-3000 provides quota write, this jira is to provide quota read.

          Thanks Kihwal Lee and Vinayakumar B!

          Show
          mingma Ming Ma added a comment - Allen Wittenauer , HDFS-3000 provides quota write, this jira is to provide quota read. Thanks Kihwal Lee and Vinayakumar B !
          Hide
          aw Allen Wittenauer added a comment -

          Argh, yeah, I missed that. Thanks.

          Just hoping someday we can undo the damage caused by that jira.

          Show
          aw Allen Wittenauer added a comment - Argh, yeah, I missed that. Thanks. Just hoping someday we can undo the damage caused by that jira.
          Hide
          zhz Zhe Zhang added a comment -

          Quick note that I'm working on backporting HDFS-10744 to branch-2.7; since this change is not in branch-2.7, the overlapping part won't be backported.

          If someone plans to backport this to branch-2.7, please remember to add the optimization from HDFS-10744.

          Show
          zhz Zhe Zhang added a comment - Quick note that I'm working on backporting HDFS-10744 to branch-2.7; since this change is not in branch-2.7, the overlapping part won't be backported. If someone plans to backport this to branch-2.7, please remember to add the optimization from HDFS-10744 .

            People

            • Assignee:
              mingma Ming Ma
              Reporter:
              jrottinghuis Joep Rottinghuis
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development