Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8763 Combine MVCC and SeqId
  3. HBASE-11135

Change region sequenceid generation so happens earlier in the append cycle rather than just before added to file

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.99.0
    • wal
    • None

    Description

      Currently we assign the region edit/sequence id just before we put it in the WAL. We do it in the single thread that feeds from the ring buffer. Doing it at this point, we can ensure order, that the edits will be in the file in accordance w/ the ordering of the region sequence id.

      But the point at which region sequence id is assigned an edit is deep down in the WAL system and there is a lag between our putting an edit into the WAL system and the edit actually getting its edit/sequence id.

      This lag – "late-binding" – complicates the unification of mvcc and region sequence id, especially around async WAL writes (and, related, for no-WAL writes) – the parent for this issue (For async, how you get the edit id in our system when the threads have all gone home – unless you make them wait?)

      Chatting w/ Jeffrey Zhong yesterday, we came up with a crazypants means of getting the region sequence id near-immediately. We'll run two ringbuffers. The first will mesh all handler threads and the consumer will generate ids (we will have order on other side of this first ring buffer), and then if async or no sync, we will just let the threads return ... updating mvcc just before we let them go. All other calls will go up on to the second ring buffer to be serviced as now (batching, distribution out among the sync'ing threads). The first rb will have no friction and should turn at fast rates compared to the second. There should not be noticeable slowdown nor do I foresee this refactor intefering w/ our multi-WAL plans.

      Attachments

        1. 11135v8.addendum.doc.txt
          2 kB
          Michael Stack
        2. 11135v8.txt
          92 kB
          Michael Stack
        3. 11135v8.txt
          92 kB
          Michael Stack
        4. 11135v7.txt
          4 kB
          Michael Stack
        5. 11135v6.txt
          92 kB
          Michael Stack
        6. 11135v5.txt
          86 kB
          Michael Stack
        7. 11135v5.txt
          86 kB
          Michael Stack
        8. 11135v5.txt
          86 kB
          Michael Stack
        9. 11135v2.txt
          81 kB
          Michael Stack
        10. 11135.wip.txt
          62 kB
          Michael Stack

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            stack Michael Stack
            stack Michael Stack
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment