Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-19064

Synchronous replication for HBase

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: Replication
    • Labels:
      None

      Description

      The guys from Alibaba made a presentation on HBaseCon Asia about the synchronous replication for HBase. We(Xiaomi) think this is a very useful feature for HBase so we want to bring it into the community version.

      This is a big feature so we plan to do it in a feature branch.

        Attachments

          Issue Links

          1.
          Add a remote peer cluster wal directory config for synchronous replication Sub-task Resolved Guanghao Zhang
          2.
          Support setting up two clusters with A and S state Sub-task Resolved Duo Zhang
          3.
          Implement a procedure to convert RS from A to DA Sub-task Resolved Unassigned
          4.
          Implement a procedure to convert RS from S to DA Sub-task Resolved Guanghao Zhang
          5.
          Reject read/write from client but accept write from replication in state S Sub-task Resolved Duo Zhang
          6.
          Introduce a new log writer which can write to two HDFSes Sub-task Resolved Duo Zhang
          7.
          Introduce a special WALProvider for synchronous replication Sub-task Resolved Duo Zhang
          8.
          Add a new cluster state flag for synchronous replication Sub-task Resolved Guanghao Zhang
          9.
          Reject the replication request when peer is DA or A state Sub-task Resolved Zheng Hu
          10.
          Allow temporarily skipping the creation of remote writer for SynchronousReplicationWALProvider Sub-task Resolved Unassigned
          11.
          Complete the procedure for adding a sync replication peer Sub-task Resolved Duo Zhang
          12.
          Use protobuf instead of enum.ordinal to store SyncReplicationState Sub-task Resolved Guanghao Zhang
          13.
          Add UT for sync replication peer in DA state Sub-task Resolved Duo Zhang
          14.
          Only allow table replication for sync replication for now Sub-task Resolved Guanghao Zhang
          15.
          Introduce a new base class for replication peer procedure Sub-task Resolved Duo Zhang
          16.
          Allow write request from replication but reject write request from user client when S state. Sub-task Resolved Zheng Hu
          17.
          Only allow removing sync replication peer which is in DA state Sub-task Resolved Zheng Hu
          18.
          General framework to transit sync replication state Sub-task Resolved Duo Zhang
          19.
          General framework to transit sync replication state Sub-task Resolved Unassigned
          20.
          Implement a procedure to replay sync replication wal for standby cluster Sub-task Resolved Guanghao Zhang
          21.
          Create remote wal directory when transitting to state S Sub-task Resolved Duo Zhang
          22.
          Remove the SYNC_REPLICATION_ENABLED flag Sub-task Resolved Guanghao Zhang
          23.
          Forbid major compaction when standby cluster replay the remote wals Sub-task Resolved Guanghao Zhang
          24.
          Also remove the wal file in remote cluster when we finish replicating a file Sub-task Resolved Duo Zhang
          25.
          Operational documents for synchronous replication. Sub-task Patch Available Zheng Hu
          26.
          Do not write the cluster id of the current active cluster when writing remote WAL Sub-task Resolved Zheng Hu
          27.
          Give up replicating anything in S state Sub-task Resolved Duo Zhang
          28.
          Cleanup related resources when remove a sync replication peer Sub-task Resolved Zheng Hu
          29.
          Also remove remote wals when peer is in DA state Sub-task Resolved Duo Zhang
          30.
          Support removing a ReplicationSourceShipper for a special wal group Sub-task Resolved Duo Zhang
          31.
          Support removing a WAL from LogRoller Sub-task Resolved Guanghao Zhang
          32.
          Concurrency issue for CombinedAsyncWriter Sub-task Resolved Unassigned
          33.
          NPE in RecoverStandbyProcedure.execute Sub-task Resolved Guanghao Zhang
          34.
          Check remote WAL directory when creating peer and transiting peer to A Sub-task Resolved Duo Zhang
          35.
          Allow writing WAL to local and remote cluster concurrently Sub-task Resolved Duo Zhang
          36.
          Polish the WAL switching when transiting from A to S Sub-task Resolved Duo Zhang
          37.
          Reopen regions using ReopenTableRegionsProcedure Sub-task Resolved Duo Zhang
          38.
          Performance comparison of synchronous replication branch and master branch Sub-task Open Zheng Hu

            Activity

              People

              • Assignee:
                Apache9 Duo Zhang
                Reporter:
                Apache9 Duo Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated: