HBase
  1. HBase
  2. HBASE-10805

Speed up KeyValueHeap.next() a bit

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.1, 0.99.0, 0.94.19, 0.96.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      See discussion on HBASE-9969.
      This was brought up by Matt Corgan. Even though I didn't believe him first.

            KeyValueScanner topScanner = this.heap.peek();
            if (topScanner == null ||
                this.comparator.compare(kvNext, topScanner.peek()) >= 0) {
              this.heap.add(this.current);
              this.current = pollRealKV();
            }
      

      We already have the invariant everywhere this.current always has a real KV polled. So adding current back to the heap followed by pollRealKV() is a no-op if this.current is the only scanner anyway.

      This can be changed to:

            ...
            if (topScanner != null &&
                this.comparator.compare(kvNext, topScanner.peek()) >= 0) {
            ...
      

      With 20m rows, 5 cols each, everything in the cache, fully compacted - i.e. one HFile per store, a scan that filters everything at the server through all 100m KVs takes 15.1s without the change and 13.3s with it, so a 12% improvement.

      1. 10805-trunk.txt
        0.8 kB
        Lars Hofhansl

        Activity

        Lars Hofhansl made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Andrew Purtell made changes -
        Fix Version/s 0.98.1 [ 12325664 ]
        Fix Version/s 0.98.2 [ 12326505 ]
        Lars Hofhansl made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Assignee Lars Hofhansl [ lhofhansl ]
        Resolution Fixed [ 1 ]
        Lars Hofhansl made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Lars Hofhansl made changes -
        Field Original Value New Value
        Attachment 10805-trunk.txt [ 12635957 ]
        Lars Hofhansl created issue -

          People

          • Assignee:
            Lars Hofhansl
            Reporter:
            Lars Hofhansl
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development