Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-1177

Delay when client is located on the same node as the regionserver



    • Bug
    • Status: Closed
    • Major
    • Resolution: Invalid
    • 0.19.0
    • None
    • Performance
    • None
    • Linux 2.6.25 x86_64


      During testing of HBASE-80, we uncovered a strange 40ms delay for random reads. We ran a series of tests and found that it only happens when the client is on the same node as the RS and for a certain range of payloads (not specifically related to number of columns or size of them, only total payload). It appears to be precisely 40ms every time.

      Unsure if this is particular to our architecture, but it does happen on all nodes we've tried. Issue completely goes away with very large payloads or moving the client.

      Will post a test program tomorrow if anyone can test on a different architecture.

      Making a blocker for 0.20. Since this happens when you have an MR task running local to the RS, and this is what we try to do, might also consider making this a blocker for 0.19.1.


        1. ReadDelayTest.java
          4 kB
          Jonathan Gray
        2. screenshot-1.jpg
          141 kB
          Jim Kellerman
        3. screenshot-2.jpg
          119 kB
          Jim Kellerman
        4. screenshot-3.jpg
          82 kB
          Jonathan Gray
        5. screenshot-4.jpg
          132 kB
          Jim Kellerman
        6. getRow times.jpg
          130 kB
          Jim Kellerman
        7. Contribution of seekTo to getClosest time.jpg
          139 kB
          Jim Kellerman
        8. Contribution of next to getRow time.jpg
          128 kB
          Jim Kellerman
        9. Contribution of getClosest to getRow time.jpg
          166 kB
          Jim Kellerman
        10. getRow + round-trip vs # columns.jpg
          111 kB
          Jim Kellerman
        11. zoom of columns vs round-trip blowup.jpg
          94 kB
          Jim Kellerman
        12. Elapsed time of RowResults.readFields.jpg
          105 kB
          Jim Kellerman
        13. RowResults.readFields zoomed.jpg
          101 kB
          Jim Kellerman

        Issue Links



              Unassigned Unassigned
              streamy Jonathan Gray
              0 Vote for this issue
              4 Start watching this issue