Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-5979

Non-pread DFSInputStreams should be associated with scanners, not HFile.Readers

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Performance, regionserver
    • Labels:
      None

      Description

      Currently, every HFile.Reader has a single DFSInputStream, which it uses to service all gets and scans. For gets, we use the positional read API (aka "pread") and for scans we use a synchronized block to seek, then read. The advantage of pread is that it doesn't hold any locks, so multiple gets can proceed at the same time. The advantage of seek+read for scans is that the datanode starts to send the entire rest of the HDFS block, rather than just the single hfile block necessary. So, in a single thread, pread is faster for gets, and seek+read is faster for scans since you get a strong pipelining effect.

      However, in a multi-threaded case where there are multiple scans (including scans which are actually part of compactions), the seek+read strategy falls apart, since only one scanner may be reading at a time. Additionally, a large amount of wasted IO is generated on the datanode side, and we get none of the earlier-mentioned advantages.

      In one test, I switched scans to always use pread, and saw a 5x improvement in throughput of the YCSB scan-only workload, since it previously was completely blocked by contention on the DFSIS lock.

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tlipcon Todd Lipcon

              Dates

              • Created:
                Updated:

                Issue deployment