Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7347 Allow multiple readers per storefile
  3. HBASE-7336

HFileBlock.readAtOffset does not work well with multiple threads

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.94.4, 0.95.0
    • Component/s: Performance
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Improves read concurrency by changing HFileBlock#readAtOffset from synchronized to reentrant lock; will have contending scanners fall back to preading rather than synchronized seek+read. Keeps all cores busy rather than just the one.

      Description

      HBase grinds to a halt when many threads scan along the same set of blocks and neither read short circuit is nor block caching is enabled for the dfs client ... disabling the block cache makes sense on very large scans.

      It turns out that synchronizing in istream in HFileBlock.readAtOffset is the culprit.

        Attachments

        1. 7336-0.96.txt
          3 kB
          Lars Hofhansl
        2. 7336-0.94.txt
          3 kB
          Lars Hofhansl

          Issue Links

            Activity

              People

              • Assignee:
                lhofhansl Lars Hofhansl
                Reporter:
                lhofhansl Lars Hofhansl
              • Votes:
                0 Vote for this issue
                Watchers:
                14 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: