Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-10015

Replace intrinsic locking with explicit locks in StoreScanner

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.96.1, 0.94.15
    • Component/s: None
    • Labels:
    • Hadoop Flags:
      Reviewed

      Description

      Did some more profiling (this time with a sampling profiler) and StoreScanner.peek() showed up a lot in the samples. At first that was surprising, but peek is synchronized, so it seems a lot of the sync'ing cost is eaten there.
      It seems the only reason we have to synchronize all these methods is because a concurrent flush or compaction can change the scanner stack, other than that only a single thread should access a StoreScanner at any given time.
      So replaced updateReaders() with some code that just indicates to the scanner that the readers should be updated and then make it the using thread's responsibility to do the work.
      The perf improvement from this is staggering. I am seeing somewhere around 3x scan performance improvement across all scenarios.

      Now, the hard part is to reason about whether this is 100% correct. I ran TestAtomicOperation and TestAcidGuarantees a few times in a loop, all still pass.

      Will attach a sample patch.

        Attachments

        1. 10015-0.94.txt
          5 kB
          Lars Hofhansl
        2. TestLoad.java
          4 kB
          Lars Hofhansl
        3. 10015-0.94-withtest.txt
          8 kB
          Lars Hofhansl
        4. 10015-trunk.txt
          5 kB
          Lars Hofhansl
        5. 10015-0.94-v2.txt
          6 kB
          Lars Hofhansl
        6. 10015-trunk-v2.txt
          5 kB
          Lars Hofhansl
        7. 10015-0.94-v3.txt
          4 kB
          Lars Hofhansl
        8. 10015-trunk-v3.txt
          4 kB
          Lars Hofhansl
        9. 10015-0.94-v4.txt
          4 kB
          Lars Hofhansl
        10. 10015-trunk-v4.txt
          4 kB
          Lars Hofhansl
        11. 10015-trunk-v4.txt
          4 kB
          stack
        12. 10015-trunk-v4.txt
          4 kB
          stack
        13. 10015-0.94-new-sample.txt
          9 kB
          Lars Hofhansl
        14. 10015-0.94-lock.txt
          4 kB
          Lars Hofhansl
        15. 10015-trunk-lock.txt
          4 kB
          Lars Hofhansl

          Activity

            People

            • Assignee:
              lhofhansl Lars Hofhansl
              Reporter:
              lhofhansl Lars Hofhansl
            • Votes:
              1 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: