HBase
  1. HBase
  2. HBASE-10275

[89-fb] Guarantee the sequenceID in each Region is strictly monotonic increasing

    Details

    • Type: New Feature New Feature
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      HBASE-8741 has implemented the per-region sequence ID. It would be even better to guarantee that the sequencing is strictly monotonic increasing so that HLog-Based Async Replication is able to delivery transactions in order in the case of region movements.

        Issue Links

          Activity

          Hide
          Liyin Tang added a comment -

          HBASE-10343 might resolve this issue in much easier way.

          Show
          Liyin Tang added a comment - HBASE-10343 might resolve this issue in much easier way.
          Hide
          Liyin Tang added a comment -

          The problem you have described is exactly what we want to resolve. Basically if the sequenceID for each region is strictly monotonic increasing, then in the case of a region moving from A to B, the replication stream in B would know the gap/lag for that region in the previous replication stream A.

          As you mentioned but slightly different: The fix is to guarantee the old hlog entries of a region from the previous region server been fully replicated, before starting to replicate this region from a new region server.

          Show
          Liyin Tang added a comment - The problem you have described is exactly what we want to resolve. Basically if the sequenceID for each region is strictly monotonic increasing, then in the case of a region moving from A to B, the replication stream in B would know the gap/lag for that region in the previous replication stream A. As you mentioned but slightly different: The fix is to guarantee the old hlog entries of a region from the previous region server been fully replicated, before starting to replicate this region from a new region server.
          Hide
          Honghua Feng added a comment -

          For your reference, HBASE-9465 describes the problem of no guarantee of serial transaction delivery to peer in case failover or region-move.
          In essence, it's hard to fix if we don't synchronize the previous(or worker regionserver which takes over the hlog pushing for the failed regionserver) and current hosting regionserver on hlog push. Without synchronization two different regionservers can push hlog entries of a same region with different pace. Another alternative fix is to guarantee the old hlog entries of a region have all been pushed to peer before it can be opened by a new regionserver.

          Show
          Honghua Feng added a comment - For your reference, HBASE-9465 describes the problem of no guarantee of serial transaction delivery to peer in case failover or region-move. In essence, it's hard to fix if we don't synchronize the previous(or worker regionserver which takes over the hlog pushing for the failed regionserver) and current hosting regionserver on hlog push. Without synchronization two different regionservers can push hlog entries of a same region with different pace. Another alternative fix is to guarantee the old hlog entries of a region have all been pushed to peer before it can be opened by a new regionserver.
          Hide
          Honghua Feng added a comment -

          To achieve the goal of in-order (hlog) transaction delivery, also need to guarantee all the older(smaller) hlog entries in previous regionserver have been successfully pushed(replicated) to peer before the region is served by the new regionserver, right? otherwise it's still possible the hlog entries with smaller sequenceid are pushed(replicated) to peer in previous hosting regionserver after the ones with greater sequenceid in the new/current hosting regionserver, right?

          For region movement in case of regionserver failover(if we deem it another kind of region movement, though passively), the hlog files containing un-pushed entries for the region will be handled by a different regionserver other than the region's new hosting regionserver, under this situation, it needs the communication/synchronization between these two regionservers to achieve the region's in-order transaction delivery from the overall perspective.

          Show
          Honghua Feng added a comment - To achieve the goal of in-order (hlog) transaction delivery, also need to guarantee all the older(smaller) hlog entries in previous regionserver have been successfully pushed(replicated) to peer before the region is served by the new regionserver, right? otherwise it's still possible the hlog entries with smaller sequenceid are pushed(replicated) to peer in previous hosting regionserver after the ones with greater sequenceid in the new/current hosting regionserver, right? For region movement in case of regionserver failover(if we deem it another kind of region movement, though passively), the hlog files containing un-pushed entries for the region will be handled by a different regionserver other than the region's new hosting regionserver, under this situation, it needs the communication/synchronization between these two regionservers to achieve the region's in-order transaction delivery from the overall perspective.

            People

            • Assignee:
              Liyin Tang
              Reporter:
              Liyin Tang
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:

                Development