Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: caching
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Add per-cache-pool default replication num configuration

      Description

      For now, hdfs cacheadmin can not set a default replication num for cached directive in the same cachepool. Each cache directive added in the same cache pool should set their own replication num individually.

      Consider this situation, we add daily hive table into cache pool "hive" .Each time i should set the same replication num for every table directive in the same cache pool.

      I think we should enable setting a default replication num for a cachepool that every cache directive in the pool can inherit replication configuration from the pool. Also cache directive can override replication configuration explicitly by calling "add & modify directive -replication" command from cli.

      1. HDFS-10328.004.patch
        24 kB
        Colin P. McCabe
      2. HDFS-10328.003.patch
        23 kB
        xupeng
      3. HDFS-10328.002.patch
        24 kB
        xupeng
      4. HDFS-10328.001.patch
        23 kB
        xupeng

        Issue Links

          Activity

          Hide
          xupener xupeng added a comment -

          Attach a patch.

          User can set replication for pool by using "cacheadmin -addPool -replication" or "cacheadmin -modifyPool -replication". When adding a cache directive into pool without setting replication num, the cache directive will inherit replication configuration from cache pool.

          Also you can override this setting by using “cacheadmin -addDirective -replication” or "cacheadmin -modifyDirective -replication".

          Show
          xupener xupeng added a comment - Attach a patch. User can set replication for pool by using "cacheadmin -addPool -replication" or "cacheadmin -modifyPool -replication". When adding a cache directive into pool without setting replication num, the cache directive will inherit replication configuration from cache pool. Also you can override this setting by using “cacheadmin -addDirective -replication” or "cacheadmin -modifyDirective -replication".
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 0s Docker mode activated.
          -1 docker 0m 3s Docker failed to build yetus/hadoop:fbe3e86.



          Subsystem Report/Notes
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800713/HDFS-10328.001.patch
          JIRA Issue HDFS-10328
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15291/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 0s Docker mode activated. -1 docker 0m 3s Docker failed to build yetus/hadoop:fbe3e86. Subsystem Report/Notes JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12800713/HDFS-10328.001.patch JIRA Issue HDFS-10328 Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15291/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          xupener xupeng added a comment -

          Hi Colin P. McCabe Andrew Wang

          could you please review this issue and patch , thanks

          Show
          xupener xupeng added a comment - Hi Colin P. McCabe Andrew Wang could you please review this issue and patch , thanks
          Hide
          cmccabe Colin P. McCabe added a comment -

          Hi xupeng,

          Interesting idea. However, this doesn't sound like "cache pool level replication management", since the replication management is still per-directive, even after this patch. This seems like adding a per-cache-pool default. If you agree, can you update the JIRA name and some of the names in the patch?

          Show
          cmccabe Colin P. McCabe added a comment - Hi xupeng , Interesting idea. However, this doesn't sound like "cache pool level replication management", since the replication management is still per-directive, even after this patch. This seems like adding a per-cache-pool default. If you agree, can you update the JIRA name and some of the names in the patch?
          Hide
          xupener xupeng added a comment -

          Hi Colin P. McCabe

          Thanks for your reply , and sorry for my late reply.

          I agree with your opinion, i have updated the name and description of the JIRA and i will update a new patch soon.

          Show
          xupener xupeng added a comment - Hi Colin P. McCabe : Thanks for your reply , and sorry for my late reply. I agree with your opinion, i have updated the name and description of the JIRA and i will update a new patch soon.
          Hide
          xupener xupeng added a comment -

          Attach a new patch, modify cli command arg name , and change the filed name of some class.

          Show
          xupener xupeng added a comment - Attach a new patch, modify cli command arg name , and change the filed name of some class.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 32s Maven dependency ordering for branch
          +1 mvninstall 7m 36s trunk passed
          +1 compile 1m 44s trunk passed with JDK v1.8.0_92
          +1 compile 1m 29s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 26s trunk passed
          +1 mvnsite 1m 35s trunk passed
          +1 mvneclipse 0m 27s trunk passed
          +1 findbugs 3m 58s trunk passed
          +1 javadoc 1m 47s trunk passed with JDK v1.8.0_92
          +1 javadoc 3m 10s trunk passed with JDK v1.7.0_95
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 58s the patch passed
          +1 compile 2m 33s the patch passed with JDK v1.8.0_92
          +1 cc 2m 32s the patch passed
          +1 javac 2m 32s the patch passed
          +1 compile 1m 42s the patch passed with JDK v1.7.0_95
          +1 cc 1m 42s the patch passed
          +1 javac 1m 42s the patch passed
          +1 checkstyle 0m 34s hadoop-hdfs-project: patch generated 0 new + 157 unchanged - 2 fixed = 157 total (was 159)
          +1 mvnsite 1m 46s the patch passed
          +1 mvneclipse 0m 35s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 xml 0m 0s The patch has no ill-formed XML file.
          +1 findbugs 5m 25s the patch passed
          +1 javadoc 2m 13s the patch passed with JDK v1.8.0_92
          +1 javadoc 2m 24s the patch passed with JDK v1.7.0_95
          +1 unit 1m 1s hadoop-hdfs-client in the patch passed with JDK v1.8.0_92.
          -1 unit 97m 22s hadoop-hdfs in the patch failed with JDK v1.8.0_92.
          +1 unit 1m 9s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95.
          -1 unit 103m 41s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 26s Patch does not generate ASF License warnings.
          249m 23s



          Reason Tests
          JDK v1.8.0_92 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.TestHFlush
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.TestFileAppend
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.TestAsyncDFSRename
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl
            hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.TestCrcCorruption
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.TestAsyncDFSRename



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:cf2ee45
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801918/HDFS-10328.002.patch
          JIRA Issue HDFS-10328
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux 6980cca59e97 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / c1cc6ac
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_92.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15345/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15345/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 7m 36s trunk passed +1 compile 1m 44s trunk passed with JDK v1.8.0_92 +1 compile 1m 29s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 26s trunk passed +1 mvnsite 1m 35s trunk passed +1 mvneclipse 0m 27s trunk passed +1 findbugs 3m 58s trunk passed +1 javadoc 1m 47s trunk passed with JDK v1.8.0_92 +1 javadoc 3m 10s trunk passed with JDK v1.7.0_95 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 58s the patch passed +1 compile 2m 33s the patch passed with JDK v1.8.0_92 +1 cc 2m 32s the patch passed +1 javac 2m 32s the patch passed +1 compile 1m 42s the patch passed with JDK v1.7.0_95 +1 cc 1m 42s the patch passed +1 javac 1m 42s the patch passed +1 checkstyle 0m 34s hadoop-hdfs-project: patch generated 0 new + 157 unchanged - 2 fixed = 157 total (was 159) +1 mvnsite 1m 46s the patch passed +1 mvneclipse 0m 35s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 5m 25s the patch passed +1 javadoc 2m 13s the patch passed with JDK v1.8.0_92 +1 javadoc 2m 24s the patch passed with JDK v1.7.0_95 +1 unit 1m 1s hadoop-hdfs-client in the patch passed with JDK v1.8.0_92. -1 unit 97m 22s hadoop-hdfs in the patch failed with JDK v1.8.0_92. +1 unit 1m 9s hadoop-hdfs-client in the patch passed with JDK v1.7.0_95. -1 unit 103m 41s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 26s Patch does not generate ASF License warnings. 249m 23s Reason Tests JDK v1.8.0_92 Failed junit tests hadoop.hdfs.TestBlockStoragePolicy   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.TestHFlush   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.TestFileAppend   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.TestAsyncDFSRename JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.datanode.TestDataNodeUUID   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl   hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.TestCrcCorruption   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.TestAsyncDFSRename Subsystem Report/Notes Docker Image:yetus/hadoop:cf2ee45 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12801918/HDFS-10328.002.patch JIRA Issue HDFS-10328 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux 6980cca59e97 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / c1cc6ac Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_92 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_92.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_92.txt https://builds.apache.org/job/PreCommit-HDFS-Build/15345/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15345/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15345/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks for the patch, xupeng.

          diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto
          index 7acb394..73db055 100644
          --- a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto
          +++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto
          @@ -533,7 +533,8 @@ message CachePoolInfoProto {
             optional string groupName = 3;
             optional int32 mode = 4;
             optional int64 limit = 5;
          -  optional int64 maxRelativeExpiry = 6;
          +  optional uint32 defaultReplication = 6;
          +  optional int64 maxRelativeExpiry = 7;
           }
          

          Please be careful not to remove or change fields that already exist. In this case, you have moved maxRelativeExpiry from field 6 to field 7, which is an incompatible change. Instead, you should simply add your new field to the end.

          I suggest using something like this:

          +  optional uint32 defaultReplication = 6 [default=1];
          

          To avoid having to programmatically add a default of 1 in so many places.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks for the patch, xupeng . diff --git a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto index 7acb394..73db055 100644 --- a/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto +++ b/hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto @@ -533,7 +533,8 @@ message CachePoolInfoProto { optional string groupName = 3; optional int32 mode = 4; optional int64 limit = 5; - optional int64 maxRelativeExpiry = 6; + optional uint32 defaultReplication = 6; + optional int64 maxRelativeExpiry = 7; } Please be careful not to remove or change fields that already exist. In this case, you have moved maxRelativeExpiry from field 6 to field 7, which is an incompatible change. Instead, you should simply add your new field to the end. I suggest using something like this: + optional uint32 defaultReplication = 6 [ default =1]; To avoid having to programmatically add a default of 1 in so many places.
          Hide
          xupener xupeng added a comment -

          Hi Colin P. McCabe

          Thanks for the review.

          I followed your advice, changed CachePoolInfoProto definition. Please refer to the new patch i submitted.

          Show
          xupener xupeng added a comment - Hi Colin P. McCabe Thanks for the review. I followed your advice, changed CachePoolInfoProto definition. Please refer to the new patch i submitted.
          Hide
          xupener xupeng added a comment -

          Don't know why HDFS-10328.003.patch has not triggered Jenkins.

          Resubmit the patch with a new version.

          Show
          xupener xupeng added a comment - Don't know why HDFS-10328 .003.patch has not triggered Jenkins. Resubmit the patch with a new version.
          Hide
          cmccabe Colin P. McCabe added a comment -

          +1 pending jenkins. Thanks, xupeng.

          Show
          cmccabe Colin P. McCabe added a comment - +1 pending jenkins. Thanks, xupeng .
          Hide
          cmccabe Colin P. McCabe added a comment -

          Reposting patch 004 (and rebasing on trunk) to get a Jenkins run

          Show
          cmccabe Colin P. McCabe added a comment - Reposting patch 004 (and rebasing on trunk) to get a Jenkins run
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          0 mvndep 0m 7s Maven dependency ordering for branch
          +1 mvninstall 7m 32s trunk passed
          +1 compile 1m 41s trunk passed
          +1 checkstyle 0m 36s trunk passed
          +1 mvnsite 1m 42s trunk passed
          +1 mvneclipse 0m 26s trunk passed
          +1 findbugs 3m 33s trunk passed
          +1 javadoc 1m 21s trunk passed
          0 mvndep 0m 8s Maven dependency ordering for patch
          +1 mvninstall 1m 30s the patch passed
          +1 compile 1m 39s the patch passed
          +1 cc 1m 39s the patch passed
          +1 javac 1m 39s the patch passed
          +1 checkstyle 0m 32s the patch passed
          +1 mvnsite 1m 38s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 xml 0m 1s The patch has no ill-formed XML file.
          +1 findbugs 3m 53s the patch passed
          +1 javadoc 1m 20s the patch passed
          +1 unit 1m 2s hadoop-hdfs-client in the patch passed.
          -1 unit 60m 32s hadoop-hdfs in the patch failed.
          +1 asflicense 0m 18s The patch does not generate ASF License warnings.
          91m 49s



          Reason Tests
          Failed junit tests hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA
            hadoop.hdfs.server.namenode.TestNamenodeRetryCache
            hadoop.hdfs.server.namenode.TestCacheDirectives
            hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:e2f6409
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811444/HDFS-10328.004.patch
          JIRA Issue HDFS-10328
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml
          uname Linux b67b47cf54c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 2800695
          Default Java 1.8.0_91
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/15814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15814/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15814/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. 0 mvndep 0m 7s Maven dependency ordering for branch +1 mvninstall 7m 32s trunk passed +1 compile 1m 41s trunk passed +1 checkstyle 0m 36s trunk passed +1 mvnsite 1m 42s trunk passed +1 mvneclipse 0m 26s trunk passed +1 findbugs 3m 33s trunk passed +1 javadoc 1m 21s trunk passed 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 1m 30s the patch passed +1 compile 1m 39s the patch passed +1 cc 1m 39s the patch passed +1 javac 1m 39s the patch passed +1 checkstyle 0m 32s the patch passed +1 mvnsite 1m 38s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 1s The patch has no ill-formed XML file. +1 findbugs 3m 53s the patch passed +1 javadoc 1m 20s the patch passed +1 unit 1m 2s hadoop-hdfs-client in the patch passed. -1 unit 60m 32s hadoop-hdfs in the patch failed. +1 asflicense 0m 18s The patch does not generate ASF License warnings. 91m 49s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.ha.TestRetryCacheWithHA   hadoop.hdfs.server.namenode.TestNamenodeRetryCache   hadoop.hdfs.server.namenode.TestCacheDirectives   hadoop.hdfs.tools.offlineEditsViewer.TestOfflineEditsViewer Subsystem Report/Notes Docker Image:yetus/hadoop:e2f6409 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12811444/HDFS-10328.004.patch JIRA Issue HDFS-10328 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc xml uname Linux b67b47cf54c2 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 2800695 Default Java 1.8.0_91 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/15814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/15814/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/15814/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs-client hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project Console output https://builds.apache.org/job/PreCommit-HDFS-Build/15814/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9987 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9987/)
          HDFS-10328. Add per-cache-pool default replication num configuration (cmccabe: rev 5f6b4157a40e974ccc6a56c39dbd35c54f393fbd)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CacheAdmin.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CacheManager.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCacheAdminConf.xml
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CachePool.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolInfo.java
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9987 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9987/ ) HDFS-10328 . Add per-cache-pool default replication num configuration (cmccabe: rev 5f6b4157a40e974ccc6a56c39dbd35c54f393fbd) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/CacheAdmin.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSImageSerialization.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CacheManager.java hadoop-hdfs-project/hadoop-hdfs/src/test/resources/testCacheAdminConf.xml hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/CachePool.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/CachePoolInfo.java hadoop-hdfs-project/hadoop-hdfs-client/src/main/proto/ClientNamenodeProtocol.proto hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocolPB/PBHelperClient.java
          Hide
          cmccabe Colin P. McCabe added a comment -

          Committed to 2.9. Thanks, xupeng.

          Show
          cmccabe Colin P. McCabe added a comment - Committed to 2.9. Thanks, xupeng .
          Hide
          kshukla Kuhu Shukla added a comment -

          xupeng, Colin P. McCabe, it seems that TestRetryCacheWithHA , TestNamenodeRetryCache, TestCacheDirectives, TestOfflineEditsViewer
          are failing after this commit. Should I open separate JIRA for these? Thanks!

          Show
          kshukla Kuhu Shukla added a comment - xupeng , Colin P. McCabe , it seems that TestRetryCacheWithHA , TestNamenodeRetryCache, TestCacheDirectives, TestOfflineEditsViewer are failing after this commit. Should I open separate JIRA for these? Thanks!
          Hide
          cmccabe Colin P. McCabe added a comment -

          Sorry for the breakage, Kuhu Shukla. HDFS-10555 should have fixed it-- check it out.

          Show
          cmccabe Colin P. McCabe added a comment - Sorry for the breakage, Kuhu Shukla . HDFS-10555 should have fixed it-- check it out.
          Hide
          xupener xupeng added a comment -

          Colin P. McCabe

          Thanks a lot for the review

          Show
          xupener xupeng added a comment - Colin P. McCabe Thanks a lot for the review

            People

            • Assignee:
              xupener xupeng
              Reporter:
              xupener xupeng
            • Votes:
              1 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development