Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-2248

Provide new non-copy mechanism to assure atomic reads in get and scan

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.20.3
    • 0.20.4
    • None
    • None
    • Incompatible change, Reviewed
    • Hide
      This patch changes the Get code path to instead be a Scan of one row. This means than inserting cells out of timestamp order should work now (tests to verify to follow part of hbase-2294) but also that a delete at an explicit timestamp now overshadows EVEN if the effected cell is put after the delete (The old Get code path did early-out so a subsequent puts would not see the delete).
      Show
      This patch changes the Get code path to instead be a Scan of one row. This means than inserting cells out of timestamp order should work now (tests to verify to follow part of hbase-2294) but also that a delete at an explicit timestamp now overshadows EVEN if the effected cell is put after the delete (The old Get code path did early-out so a subsequent puts would not see the delete).

    Description

      HBASE-2037 introduced a new MemStoreScanner which triggers a ConcurrentSkipListMap.buildFromSorted clone of the memstore and snapshot when starting a scan.

      After upgrading to 0.20.3, we noticed a big slowdown in our use of short scans. Some of our data repesent a time series. The data is stored in time series order, MR jobs often insert/update new data at the end of the series, and queries usually have to pick up some or all of the series. These are often scans of 0-100 rows at a time. To load one page, we'll observe about 20 such scans being triggered concurrently, and they take 2 seconds to complete. Doing a thread dump of a region server shows many threads in ConcurrentSkipListMap.biuldFromSorted which traverses the entire map of key values to copy it.

      Attachments

        1. ASF.LICENSE.NOT.GRANTED--HBASE-2248-no-row-locks.txt
          4 kB
          ryan rawson
        2. ASF.LICENSE.NOT.GRANTED--HBASE-2248-rr-final1.txt
          106 kB
          ryan rawson
        3. ASF.LICENSE.NOT.GRANTED--HBASE-2248-rr-pre-durability4.txt
          126 kB
          ryan rawson
        4. ASF.LICENSE.NOT.GRANTED--put_call_graph.png
          127 kB
          Andrew Kyle Purtell
        5. ASF.LICENSE.NOT.GRANTED--profile.png
          183 kB
          Andrew Kyle Purtell
        6. HBASE-2248-GetsAsScans3.patch
          190 kB
          Michael Stack
        7. hbase-2248.txt
          5 kB
          Todd Lipcon
        8. readownwrites-lost.2.patch
          3 kB
          Todd Lipcon
        9. readownwrites-lost.patch
          3 kB
          Todd Lipcon
        10. HBASE-2248.patch
          11 kB
          Michael Stack
        11. HBASE-2248-demonstrate-previous-impl-bugs.patch
          14 kB
          Michael Stack
        12. Screen shot 2010-02-23 at 10.33.38 AM.png
          70 kB
          Michael Stack
        13. hbase-2248.gc
          67 kB
          Dave Latham
        14. threads.txt
          12 kB
          Dave Latham

        Issue Links

          Activity

            People

              ryanobjc ryan rawson
              davelatham Dave Latham
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: