Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-6522

Expose locks and leases to Coprocessors

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently it is not possible for CP to implement any of checkAndMutate type operations, because coprocessor have no way create a lock, because getLock is private HRegion (interestingly ReleaseLock is public).

      In addition it would nice if Coprocessor could hook into the RegionServers' Lease management.

      Here I propose two trivial changes:

      1. Make HRegion.getLock public
      2. Add
        Leases getLeases()

        to RegionServerServices (and hence to HRegionServer)

      1. 6522-v2.txt
        6 kB
        Lars Hofhansl
      2. 6522.txt
        2 kB
        Lars Hofhansl

        Activity

        Hide
        lhofhansl Lars Hofhansl added a comment -

        Trivial patch that does just that.

        Show
        lhofhansl Lars Hofhansl added a comment - Trivial patch that does just that.
        Hide
        stack stack added a comment -

        The getLeases has to be added to RegionServerServices? Otherwise +1 on patch.

        Show
        stack stack added a comment - The getLeases has to be added to RegionServerServices? Otherwise +1 on patch.
        Hide
        lhofhansl Lars Hofhansl added a comment -

        @Stack: Yep, because that is all a RegionObserver can see (through the RegionCoprocessorEnvironment). I think it also makes sense, because it is a service provided by the RegionServer.

        Show
        lhofhansl Lars Hofhansl added a comment - @Stack: Yep, because that is all a RegionObserver can see (through the RegionCoprocessorEnvironment). I think it also makes sense, because it is a service provided by the RegionServer.
        Hide
        stack stack added a comment -

        @Lars OK boss

        Show
        stack stack added a comment - @Lars OK boss
        Hide
        lhofhansl Lars Hofhansl added a comment -

        V2 also fixes up the mock implementations of RSServices and add some simple test code
        (which just makes sure that getLeases() and getLock(...) is accessible and can be called from a RegionObserver implementation).

        Show
        lhofhansl Lars Hofhansl added a comment - V2 also fixes up the mock implementations of RSServices and add some simple test code (which just makes sure that getLeases() and getLock(...) is accessible and can be called from a RegionObserver implementation).
        Hide
        lhofhansl Lars Hofhansl added a comment -

        BTW. this also came up in an internal Salesforce discussion and we need something the leases for some internal use cases.

        Show
        lhofhansl Lars Hofhansl added a comment - BTW. this also came up in an internal Salesforce discussion and we need something the leases for some internal use cases.
        Hide
        lhofhansl Lars Hofhansl added a comment -

        Going to commit soon if nobody objects to the new API(s) exposed to CPs.

        Show
        lhofhansl Lars Hofhansl added a comment - Going to commit soon if nobody objects to the new API(s) exposed to CPs.
        Hide
        ram_krish ramkrishna.s.vasudevan added a comment -

        +1.

        Show
        ram_krish ramkrishna.s.vasudevan added a comment - +1.
        Hide
        lhofhansl Lars Hofhansl added a comment -

        Committed to 0.94 and 0.96.

        One part I do notice now 0.94 that Leases.createLease does not take a experation time (but it does in 0.96).
        So I would like to add an addendum here and add that to 0.94 as well.

        I.e. change

         public void createLease(String leaseName, final LeaseListener listener) 

        to the 0.96 version:

         public void createLease(String leaseName, int leaseTimeoutPeriod, final LeaseListener listener) 
        Show
        lhofhansl Lars Hofhansl added a comment - Committed to 0.94 and 0.96. One part I do notice now 0.94 that Leases.createLease does not take a experation time (but it does in 0.96). So I would like to add an addendum here and add that to 0.94 as well. I.e. change public void createLease( String leaseName, final LeaseListener listener) to the 0.96 version: public void createLease( String leaseName, int leaseTimeoutPeriod, final LeaseListener listener)
        Hide
        lhofhansl Lars Hofhansl added a comment -

        At the same time we can tackle this in another jira when it turns out to actually be a problem. As it stands in 0.94 all leases take default leasePeriod configured for this region server.

        Show
        lhofhansl Lars Hofhansl added a comment - At the same time we can tackle this in another jira when it turns out to actually be a problem. As it stands in 0.94 all leases take default leasePeriod configured for this region server.
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #127 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/127/)
        HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371024)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
        • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #127 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/127/ ) HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371024) Result = FAILURE larsh : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-0.94-security #48 (See https://builds.apache.org/job/HBase-0.94-security/48/)
        HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371026)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Show
        hudson Hudson added a comment - Integrated in HBase-0.94-security #48 (See https://builds.apache.org/job/HBase-0.94-security/48/ ) HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371026) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-0.94-security-on-Hadoop-23 #7 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/7/)
        HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371026)

        Result = FAILURE
        larsh :
        Files :

        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
        • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java
        • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Show
        hudson Hudson added a comment - Integrated in HBase-0.94-security-on-Hadoop-23 #7 (See https://builds.apache.org/job/HBase-0.94-security-on-Hadoop-23/7/ ) HBASE-6522 Expose locks and leases to Coprocessors (Revision 1371026) Result = FAILURE larsh : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/coprocessor/SimpleRegionObserver.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/util/MockRegionServerServices.java
        Hide
        stack stack added a comment -

        Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl)

        Show
        stack stack added a comment - Fix up after bulk move overwrote some 0.94.2 fix versions w/ 0.95.0 (Noticed by Lars Hofhansl)

          People

          • Assignee:
            lhofhansl Lars Hofhansl
            Reporter:
            lhofhansl Lars Hofhansl
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development