HBase
  1. HBase
  2. HBASE-10421

Investigate scan performance in 0.96 and later

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not a Problem
    • Affects Version/s: 0.98.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Testing the "internal friction" of scanning in HBase (all data in the block cache, and all rows filtered at the server by a ValueFilter, so that one really measures the work HBase does internally), I found that 0.98 is almost 35% slower than 0.94.

      Scanning 50m rows (one col each, 8 byte keys, 8 byte values) takes 13.2 in 0.94.17-SNAPSHOT and 18.5s in 0.98.

      This probably came about with all the protobuf changes in 0.96.
      It should be possible to bring back to par with 0.94.

        Issue Links

          Activity

          Lars Hofhansl created issue -
          Hide
          Lars Hofhansl added a comment -

          Assigning to me in the hope that I'll get some time for this, this coming week.

          Show
          Lars Hofhansl added a comment - Assigning to me in the hope that I'll get some time for this, this coming week.
          Lars Hofhansl made changes -
          Field Original Value New Value
          Assignee Lars Hofhansl [ lhofhansl ]
          Andrew Purtell made changes -
          Fix Version/s 0.99.0 [ 12325675 ]
          Andrew Purtell made changes -
          Affects Version/s 0.98.0 [ 12323143 ]
          Hide
          Andrew Purtell added a comment -

          I found that 0.98 is almost 35% slower than 0.94.

          I wonder what the impact of HBASE-4811 is, this was first introduced in 0.98

          Show
          Andrew Purtell added a comment - I found that 0.98 is almost 35% slower than 0.94. I wonder what the impact of HBASE-4811 is, this was first introduced in 0.98
          Andrew Purtell made changes -
          Link This issue is related to HBASE-4811 [ HBASE-4811 ]
          Hide
          chunhui shen added a comment -

          HBase-4811 only add some 'if' block, and didn't change the logic of origin scan if it is a forward scan.
          I think it won't have such a big effect.

          Jean-Marc has taken a performance test for HBASE-4811:
          https://issues.apache.org/jira/browse/HBASE-4811?focusedCommentId=13801449&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13801449

          How about between 0.94 and 0.96 ?

          Show
          chunhui shen added a comment - HBase-4811 only add some 'if' block, and didn't change the logic of origin scan if it is a forward scan. I think it won't have such a big effect. Jean-Marc has taken a performance test for HBASE-4811 : https://issues.apache.org/jira/browse/HBASE-4811?focusedCommentId=13801449&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13801449 How about between 0.94 and 0.96 ?
          Hide
          Lars Hofhansl added a comment - - edited

          I redid the test and this time I compiled everything myself.
          0.98 is just as fast as 0.94 for my test (and both are slightly faster than 0.96).

          All is good.

          Show
          Lars Hofhansl added a comment - - edited I redid the test and this time I compiled everything myself. 0.98 is just as fast as 0.94 for my test (and both are slightly faster than 0.96). All is good.
          Lars Hofhansl made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Assignee Lars Hofhansl [ lhofhansl ]
          Resolution Not A Problem [ 8 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development