Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.0, 2.0.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We have support for doing an rpc against the primary region in get/scans and also new functionality for doing failover get/scan RPCs trying primary first then doing backup rpcs to secondaries when the first one times out.

      We can add functionality for doing get/scan against a particular replica_id so that we can use it in tests, or by clients in some specific situations.

      We can think about adding support for querying the replica_id from get/scan responses coming from failover rpcs so that the client can then issue a follow up request against that particular replica_id.

      1. hbase-11572_v2.patch
        17 kB
        Enis Soztutar
      2. hbase-11572.patch
        6 kB
        Jeffrey Zhong

        Activity

        Hide
        Nick Dimiduk added a comment -

        Isn't it better to leave the client ignorant of specific replicas? It makes sense for tests, but how is it helpful for prod usecases? Maybe you have an example?

        Show
        Nick Dimiduk added a comment - Isn't it better to leave the client ignorant of specific replicas? It makes sense for tests, but how is it helpful for prod usecases? Maybe you have an example?
        Hide
        Enis Soztutar added a comment -

        I think we will keep the ability to either do strong consistency reads and also timeline consistency reads from any replica. This is more like an advanced feature (main use case would still be tests) available if you want to use.

        You can use this for example to partition the reads of a read-only table between clients, or do a read-from-any, obtain the replica Id returned, then do a subsequent read from that replica kind of use cases.

        Show
        Enis Soztutar added a comment - I think we will keep the ability to either do strong consistency reads and also timeline consistency reads from any replica. This is more like an advanced feature (main use case would still be tests) available if you want to use. You can use this for example to partition the reads of a read-only table between clients, or do a read-from-any, obtain the replica Id returned, then do a subsequent read from that replica kind of use cases.
        Hide
        Jeffrey Zhong added a comment -

        I added the support for Get not in Scan. The code will be very similar but there is no use case for Scan. We can add it when there is a need. Thanks.

        Show
        Jeffrey Zhong added a comment - I added the support for Get not in Scan. The code will be very similar but there is no use case for Scan. We can add it when there is a need. Thanks.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12661970/hbase-11572.patch
        against trunk revision .
        ATTACHMENT ID: 12661970

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 lineLengths. The patch does not introduce lines longer than 100

        +1 site. The mvn site goal succeeds with this patch.

        -1 core tests. The patch failed these unit tests:
        org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster

        -1 core zombie tests. There are 1 zombie test(s):

        Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html
        Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12661970/hbase-11572.patch against trunk revision . ATTACHMENT ID: 12661970 +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 3 new or modified tests. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . The javadoc tool did not generate any warning messages. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster -1 core zombie tests . There are 1 zombie test(s): Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//artifact/patchprocess/newPatchFindbugsWarningshbase-thrift.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10449//console This message is automatically generated.
        Hide
        Enis Soztutar added a comment -

        Jeffrey, I've reviewed your patch it looks good. I've also added shell support for specifying the region_replica_id with gets and also LTT support as well for advanced users.

        Show
        Enis Soztutar added a comment - Jeffrey, I've reviewed your patch it looks good. I've also added shell support for specifying the region_replica_id with gets and also LTT support as well for advanced users.
        Hide
        Jeffrey Zhong added a comment -

        Looks good to me(+1). Thanks Enis Soztutar for the add-on stuff!

        Show
        Jeffrey Zhong added a comment - Looks good to me(+1). Thanks Enis Soztutar for the add-on stuff!
        Hide
        Enis Soztutar added a comment -

        Committed this to master and branch-1. Thanks Jeffrey.

        Show
        Enis Soztutar added a comment - Committed this to master and branch-1. Thanks Jeffrey.
        Hide
        Hudson added a comment -

        FAILURE: Integrated in HBase-TRUNK #5413 (See https://builds.apache.org/job/HBase-TRUNK/5413/)
        HBASE-11572 Add support for doing get/scans against a particular replica_id (Jeffrey Zhong) (enis: rev aeecd20373bd3810ee8430de3beec81679129614)

        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
        • hbase-shell/src/main/ruby/hbase.rb
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
        • hbase-shell/src/main/ruby/shell/commands/get.rb
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java
        • hbase-shell/src/main/ruby/hbase/table.rb
        Show
        Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5413 (See https://builds.apache.org/job/HBase-TRUNK/5413/ ) HBASE-11572 Add support for doing get/scans against a particular replica_id (Jeffrey Zhong) (enis: rev aeecd20373bd3810ee8430de3beec81679129614) hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java hbase-shell/src/main/ruby/hbase.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java hbase-shell/src/main/ruby/shell/commands/get.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java hbase-shell/src/main/ruby/hbase/table.rb
        Hide
        Hudson added a comment -

        SUCCESS: Integrated in HBase-1.0 #114 (See https://builds.apache.org/job/HBase-1.0/114/)
        HBASE-11572 Add support for doing get/scans against a particular replica_id (Jeffrey Zhong) (enis: rev 05e67eba9da1488b5b20a7844504e35a8a5e92c5)

        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java
        • hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java
        • hbase-shell/src/main/ruby/hbase/table.rb
        • hbase-shell/src/main/ruby/shell/commands/get.rb
        • hbase-shell/src/main/ruby/hbase.rb
        • hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
        Show
        Hudson added a comment - SUCCESS: Integrated in HBase-1.0 #114 (See https://builds.apache.org/job/HBase-1.0/114/ ) HBASE-11572 Add support for doing get/scans against a particular replica_id (Jeffrey Zhong) (enis: rev 05e67eba9da1488b5b20a7844504e35a8a5e92c5) hbase-client/src/main/java/org/apache/hadoop/hbase/client/Scan.java hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestRegionReplicas.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/MultiThreadedReader.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/Query.java hbase-server/src/test/java/org/apache/hadoop/hbase/util/LoadTestTool.java hbase-client/src/main/java/org/apache/hadoop/hbase/client/RpcRetryingCallerWithReadReplicas.java hbase-shell/src/main/ruby/hbase/table.rb hbase-shell/src/main/ruby/shell/commands/get.rb hbase-shell/src/main/ruby/hbase.rb hbase-client/src/main/java/org/apache/hadoop/hbase/client/Get.java
        Hide
        Enis Soztutar added a comment -

        Closing this issue after 0.99.0 release.

        Show
        Enis Soztutar added a comment - Closing this issue after 0.99.0 release.

          People

          • Assignee:
            Jeffrey Zhong
            Reporter:
            Enis Soztutar
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development