Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0.0
    • Performance
    • None
    • Reviewed
    • Changed the write pipeline order; made it more rational, easier-to-reason-about doing all updates to WA, MemStore, and mvcc while read/write rowlock is held where before we'd release after WAL append and then do sync and mvcc.

    Description

      Change how we do our write pipeline. I want to do all write pipeline ops under row lock so I lean on this fact fixing performance regression in check-and-set type operations like increment, append, and checkAnd* (see sibling issue HBASE-15082).

      To be specific, we write like this now:

      # take rowlock
      # start mvcc
      # append to WAL
      # add to memstore
      # let go of rowlock
      # sync WAL
      # in case of error: rollback memstore
      

      Instead, write like this:

      # take rowlock
      # start mvcc
      # append to WAL
      # sync WAL
      # add to memstore
      # let go of rowlock
      
      ... no need to do rollback.
      

      The old ordering was put in place because it got better performance in a time when WAL was different and before row locks were read/write (HBASE-12751).

      Testing in branch-1 shows that a reordering and skipping mvcc waits gets us back to the performance we had before we unified mvcc and sequenceid (HBASE-8763). Tests in HBASE-15046 show that at the macro level using our usual perf tools, reordering pipeline seems to cause no slowdown (see HBASE-15046). A rough compare of increments with reordered write pipeline seems to have us getting back a bunch of our performance (see tail of https://issues.apache.org/jira/browse/HBASE-15082?focusedCommentId=15111703&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15111703 and subsequent comment).

      Attachments

        1. measurements.tgz
          2 kB
          Michael Stack
        2. 15158v4.patch
          161 kB
          Michael Stack
        3. 15158v4.patch
          161 kB
          Michael Stack
        4. 15158v3.patch
          161 kB
          Michael Stack
        5. 15158v2.patch
          258 kB
          Michael Stack
        6. 15158.patch
          248 kB
          Michael Stack

        Issue Links

          Activity

            People

              stack Michael Stack
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: