Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8741

Scope sequenceid to the region rather than regionserver (WAS: Mutations on Regions in recovery mode might have same sequenceIDs)

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.1
    • Fix Version/s: 0.98.0
    • Component/s: MTTR
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Sequenceids are now per region rather than per RegionServer.

      Description

      Currently, when opening a region, we find the maximum sequence ID from all its HFiles and then set the LogSequenceId of the log (in case the later is at a small value). This works good in recovered.edits case as we are not writing to the region until we have replayed all of its previous edits.
      With distributed log replay, if we want to enable writes while a region is under recovery, we need to make sure that the logSequenceId > maximum logSequenceId of the old regionserver. Otherwise, we might have a situation where new edits have same (or smaller) sequenceIds.

      We can store region level information in the WALTrailer, than this scenario could be avoided by:
      a) reading the trailer of the "last completed" file, i.e., last wal file which has a trailer and,
      b) completely reading the last wal file (this file would not have the trailer, so it needs to be read completely).

      In future, if we switch to multi wal file, we could read the trailer for all completed WAL files, and reading the remaining incomplete files.

        Attachments

        1. HBASE-8741-trunk-v6.4.patch
          89 kB
          Himanshu Vashishtha
        2. HBASE-8741-trunk-v6.3.patch
          86 kB
          Himanshu Vashishtha
        3. HBASE-8741-trunk-v6.2.2.patch
          86 kB
          Michael Stack
        4. HBASE-8741-trunk-v6.2.2.patch
          86 kB
          Himanshu Vashishtha
        5. HBASE-8741-trunk-v6.2.1.patch
          86 kB
          Himanshu Vashishtha
        6. HBASE-8741-trunk-v6.1-rebased.patch
          86 kB
          Himanshu Vashishtha
        7. HBASE-8741-trunk-v6.patch
          86 kB
          Himanshu Vashishtha
        8. HBASE-8741-v5-again.patch
          97 kB
          Himanshu Vashishtha
        9. HBASE-8741-v5.patch
          97 kB
          Himanshu Vashishtha
        10. HBASE-8741-v4-again.patch
          98 kB
          Michael Stack
        11. HBASE-8741-v4-again.patch
          98 kB
          Himanshu Vashishtha
        12. HBASE-8741-v4.patch
          99 kB
          Himanshu Vashishtha
        13. HBASE-8741-v3.patch
          102 kB
          Himanshu Vashishtha
        14. HBASE-8741-v2.patch
          101 kB
          Himanshu Vashishtha
        15. HBASE-8741-v0.patch
          77 kB
          Himanshu Vashishtha

          Activity

            People

            • Assignee:
              v.himanshu Himanshu Vashishtha
              Reporter:
              v.himanshu Himanshu Vashishtha
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: