Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-9465

Push entries to peer clusters serially

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 1.4.0, 2.0.0
    • None
    • regionserver, Replication
    • None
    • Reviewed
    • Hide
      Now in replication we can make sure the order of pushing logs is same as the order of requests from client. Set REPLICATION_SCOPE=2 at one cf's configuration to enable this feature.
      This feature relies on zk-less assignment, and conflicts with distributed log replay. So users must set hbase.assignment.usezk and hbase.master.distributed.log.replay to false to support this feature.
      Show
      Now in replication we can make sure the order of pushing logs is same as the order of requests from client. Set REPLICATION_SCOPE=2 at one cf's configuration to enable this feature. This feature relies on zk-less assignment, and conflicts with distributed log replay. So users must set hbase.assignment.usezk and hbase.master.distributed.log.replay to false to support this feature.

    Description

      When region-move or RS failure occurs in master cluster, the hlog entries that are not pushed before region-move or RS-failure will be pushed by original RS(for region move) or another RS which takes over the remained hlog of dead RS(for RS failure), and the new entries for the same region(s) will be pushed by the RS which now serves the region(s), but they push the hlog entries of a same region concurrently without coordination.

      This treatment can possibly lead to data inconsistency between master and peer clusters:
      1. there are put and then delete written to master cluster
      2. due to region-move / RS-failure, they are pushed by different replication-source threads to peer cluster
      3. if delete is pushed to peer cluster before put, and flush and major-compact occurs in peer cluster before put is pushed to peer cluster, the delete is collected and the put remains in peer cluster

      In this scenario, the put remains in peer cluster, but in master cluster the put is masked by the delete, hence data inconsistency between master and peer clusters

      Attachments

        1. HBASE-9465.pdf
          67 kB
          Phil Yang
        2. HBASE-9465-branch-1.v4.revert.patch
          102 kB
          Sean Busbey
        3. HBASE-9465-branch-1-v1.patch
          89 kB
          Phil Yang
        4. HBASE-9465-branch-1-v1.patch
          89 kB
          Phil Yang
        5. HBASE-9465-branch-1-v2.patch
          89 kB
          Phil Yang
        6. HBASE-9465-branch-1-v3.patch
          90 kB
          Phil Yang
        7. HBASE-9465-branch-1-v4.patch
          90 kB
          Phil Yang
        8. HBASE-9465-branch-1-v4.patch
          90 kB
          Phil Yang
        9. HBASE-9465-v1.patch
          83 kB
          Phil Yang
        10. HBASE-9465-v2.patch
          85 kB
          Duo Zhang
        11. HBASE-9465-v2.patch
          85 kB
          Phil Yang
        12. HBASE-9465-v3.patch
          87 kB
          Phil Yang
        13. HBASE-9465-v4.patch
          86 kB
          Phil Yang
        14. HBASE-9465-v5.patch
          87 kB
          Phil Yang
        15. HBASE-9465-v6.patch
          87 kB
          Duo Zhang
        16. HBASE-9465-v6.patch
          87 kB
          Phil Yang
        17. HBASE-9465-v7.patch
          87 kB
          Phil Yang
        18. HBASE-9465-v7.patch
          87 kB
          Phil Yang

        Issue Links

          Activity

            People

              yangzhe1991 Phil Yang
              fenghh Honghua Feng
              Votes:
              0 Vote for this issue
              Watchers:
              29 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: