• Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0, 1.1.0
    • Component/s: None
    • Labels:
    • Hadoop Flags:


      With the async wal approach (HBASE-11568), the edits are not persisted (to wal) in the secondary region replicas. However this means that we have to deal with secondary region replica failures.

      We can seek to re-replicate the edits from primary to the secondary when the secondary region is opened in another server but this would mean to setup a replication queue again, and holding on to the wals for longer.

      Instead, we can design it so that the edits form the secondaries are not persisted to wal, and if the secondary replica fails over, it will not start serving reads until it has guaranteed that it has all the past data.

      For guaranteeing that the secondary replica has all the edits before serving reads, we can use flush and region opening markers. Whenever a region open event is seen, it writes all the files at the time of opening to wal (HBASE-11512). In case of flush, the flushed file is written as well, and the secondary replica can do a ls for the store files and pick up all the files before the seqId of the flushed file. So, in this design, the secodary replica will wait until it sees and replays a flush or region open marker from wal from primary. and then start serving. For speeding up replica opening time, we can trigger a flush to the primary whenever the secondary replica opens as an optimization.

      1. hbase-11580-addendum.patch
        2 kB
        Enis Soztutar
      2. hbase-11580_v3.patch
        142 kB
        Enis Soztutar
      3. hbase-11580_v2.patch
        142 kB
        Enis Soztutar

        Issue Links


          Enis Soztutar created issue -
          Enis Soztutar made changes -
          Field Original Value New Value
          Assignee Enis Soztutar [ enis ]
          Enis Soztutar made changes -
          Remote Link This issue links to "review (Web Link)" [ 23357 ]
          Enis Soztutar made changes -
          Attachment hbase-11580_v2.patch [ 12701535 ]
          Enis Soztutar made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Enis Soztutar made changes -
          Attachment hbase-11580_v3.patch [ 12702007 ]
          Enis Soztutar made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 2.0.0 [ 12327188 ]
          Fix Version/s 1.1.0 [ 12329043 ]
          Resolution Fixed [ 1 ]
          Sean Busbey made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Enis Soztutar made changes -
          Attachment hbase-11580-addendum.patch [ 12702640 ]
          Enis Soztutar made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]


            • Assignee:
              Enis Soztutar
              Enis Soztutar
            • Votes:
              0 Vote for this issue
              8 Start watching this issue


              • Created: