Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8151

Decode memstoreTS in HFileReaderV2 only when necessary

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      HFiles V2 store the memstoreTS of each KV.
      In many cases all the KVs in an HFile will have a memstoreTS of 0 (that is the case when at the time the HFile was written there are no KVs that were created after the oldest still active scanner - which is frequently the case).
      In that case we:

      1. do not need to decode the memstoreTS (a vlong), since we know its value is 0 and its length is 1 byte.
      2. when we compact HFiles and all of the involved files have only KVs with memstoreTS = 0 we know ahead of time that all KVs meet this condition and we do not need to store the memstoreTS in the new HFile.

      This issue will cover the first part. The performance improvement will be modest as it is fairly cheap to decode vlongs of size 1.

        Attachments

        1. 8151-0.94.txt
          3 kB
          Lars Hofhansl
        2. 8151-0.96.txt
          3 kB
          Lars Hofhansl
        3. 8151-0.96-v2.txt
          18 kB
          Lars Hofhansl
        4. 8151-0.96-v3.txt
          19 kB
          Ted Yu
        5. 8151-0.94-v2.txt
          16 kB
          Lars Hofhansl

          Activity

            People

            • Assignee:
              larsh Lars Hofhansl
              Reporter:
              larsh Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: