HBase
  1. HBase
  2. HBASE-7564

[replication] Create interfaces for manipulation of replication state

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.96.0
    • Component/s: Replication
    • Labels:
      None

      Description

      Currently ReplicationZookeeper maintains all the zookeeper state for replication. This makes the class relatively large and slightly confusing. There are three major pieces of zookeeper state maintained for replication:

      1. The state of replication (i.e. whether replication is ENABLED or DISABLED on the cluster). This is held in the state znode.

      2. The set of slave (or peer) clusters that replication needs to ship edits to. This is held in the peer znode.

      3. The replication queues that keep track of which hlog files still need to be replicated. There is one queue for each replication source/peer cluster pair.

      Splitting each of these three pieces into their own interfaces will separate the implementation from the operations needed to manipulate replication state. This will allow easier unit testing of the replication logic and more flexibility for future implementations of how replication state is stored.

      The plan is to implement these changes as a series of patches (at least one for each of the three interfaces). The state interface will be first, since it is the most easily separable from the other two pieces.

      1. ReplicationRefactor-v2.pdf
        214 kB
        Chris Trezzo
      2. ReplicationRefactor.pdf
        217 kB
        Chris Trezzo

        Issue Links

          Activity

          Hide
          Chris Trezzo added a comment -

          Attached is a brief document that contains a diagram of the new class structure for this refactor.

          Thanks!

          Show
          Chris Trezzo added a comment - Attached is a brief document that contains a diagram of the new class structure for this refactor. Thanks!
          Hide
          stack added a comment -

          Jean-Daniel Cryans Help a brother out (our Chris). What you think of his proposal? Himanshu Vashishtha What you think Himanshu?

          Show
          stack added a comment - Jean-Daniel Cryans Help a brother out (our Chris). What you think of his proposal? Himanshu Vashishtha What you think Himanshu?
          Hide
          Chris Trezzo added a comment -

          Minor update to reflect the change in class structure: ReplicationQueuesZKImpl no longer extends ReplicationQueuesClientZKImpl. Instead they both extend ReplicationStateZKBase directly.

          Show
          Chris Trezzo added a comment - Minor update to reflect the change in class structure: ReplicationQueuesZKImpl no longer extends ReplicationQueuesClientZKImpl. Instead they both extend ReplicationStateZKBase directly.
          Hide
          stack added a comment -

          Chris Trezzo Any more you want to get in for 0.95?

          Show
          stack added a comment - Chris Trezzo Any more you want to get in for 0.95?
          Hide
          Chris Trezzo added a comment -

          Stack Yes, sorry for the delay. I'll post a patch for HBASE-7567 today.

          Show
          Chris Trezzo added a comment - Stack Yes, sorry for the delay. I'll post a patch for HBASE-7567 today.
          Hide
          Jean-Daniel Cryans added a comment -

          Hey Chris Trezzo, it looks like there's only one Jira left? HBASE-8441? And it's not targeting anything whereas this current jira is on 0.95.2. Do you happen to have a patch sitting somewhere?

          Show
          Jean-Daniel Cryans added a comment - Hey Chris Trezzo , it looks like there's only one Jira left? HBASE-8441 ? And it's not targeting anything whereas this current jira is on 0.95.2. Do you happen to have a patch sitting somewhere?
          Hide
          Chris Trezzo added a comment -

          Jean-Daniel Cryans Will attach it to HBASE-8441. Thanks! Sorry for the delay.

          Show
          Chris Trezzo added a comment - Jean-Daniel Cryans Will attach it to HBASE-8441 . Thanks! Sorry for the delay.
          Hide
          stack added a comment -

          Chris Trezzo You finished then? All the sub-issues are committed. Does that mean this is done? If so, congrats (and resolve this)! Otherwise, whats in the way?

          Show
          stack added a comment - Chris Trezzo You finished then? All the sub-issues are committed. Does that mean this is done? If so, congrats (and resolve this)! Otherwise, whats in the way?
          Hide
          Chris Trezzo added a comment -

          All done! Thanks Stack and Jean-Daniel Cryans for all the comments/reviews. Always super helpful!

          Show
          Chris Trezzo added a comment - All done! Thanks Stack and Jean-Daniel Cryans for all the comments/reviews. Always super helpful!

            People

            • Assignee:
              Chris Trezzo
              Reporter:
              Chris Trezzo
            • Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development