Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9426

Make custom distributed barrier procedure pluggable

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.2, 0.94.11
    • Fix Version/s: 0.99.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      This patch adds two new API calls to the protobuf rpc interface.

      Description

      Currently if one wants to implement a custom distributed barrier procedure (e.g., distributed log roll or distributed table flush), the HBase core code needs to be modified in order for the procedure to work.

      Looking into the snapshot code (especially on region server side), most of the code to enable the procedure are generic life-cycle management (i.e., init, start, stop). We can make this part pluggable.

      Here is the proposal. Following the coprocessor example, we define two properties:

      hbase.procedure.regionserver.classes
      hbase.procedure.master.classes
      

      The values for both are comma delimited list of classes. On region server side, the classes implements the following interface:

      public interface RegionServerProcedureManager {
        public void initialize(RegionServerServices rss) throws KeeperException;
        public void start();
        public void stop(boolean force) throws IOException;
        public String getProcedureName();
      }
      

      While on Master side, the classes implement the interface:

      public interface MasterProcedureManager {
        public void initialize(MasterServices master) throws KeeperException, IOException, UnsupportedOperationException;
        public void stop(String why);
        public String getProcedureName();
        public void execProcedure(ProcedureDescription desc) throws IOException;
        IOException;
      }
      

      Where the ProcedureDescription is defined as

      message ProcedureDescription {
        required string name = 1;
        required string instance = 2;
        optional int64 creationTime = 3 [default = 0];
        message Property {
          required string tag = 1;
          optional string value = 2;
        }
        repeated Property props = 4;
      }
      

      A generic API can be defined on HMaster to trigger a procedure:

      public boolean execProcedure(ProcedureDescription desc) throws IOException;
      

      SnapshotManager and RegionServerSnapshotManager are special examples of MasterProcedureManager and RegionServerProcedureManager. They will be automatically included (users don't need to specify them in the conf file).

        Attachments

        1. HBASE-9426.patch.1
          18 kB
          Richard Ding
        2. HBASE-9426.patch.2
          225 kB
          Richard Ding
        3. HBASE-9426.patch.3
          53 kB
          Richard Ding
        4. HBASE-9426-4.patch
          239 kB
          Richard Ding
        5. HBASE-9426-4.patch
          239 kB
          Richard Ding
        6. HBASE-9426-6.patch
          259 kB
          Richard Ding
        7. HBASE-9426-7.patch
          259 kB
          Richard Ding

          Issue Links

            Activity

              People

              • Assignee:
                rding Richard Ding
                Reporter:
                rding Richard Ding
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: