Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-3927

display total uncompressed byte size of a region in web UI

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 0.92.0
    • metrics
    • None
    • Reviewed

    Description

      The decision to split data blocks when flushing and compacting is made based on the uncompressed data size which can often lead to compressed disk blocks that are a fraction of the intended 64 KB (default). This often leads to a larger number of blocks and index entries than expected and can cause block indexes to take up GB of memory.

      There is already a "long totalUncompressedBytes" written to the HFileTrailer. It would be nice to expose this in the web UI to make it easier to calculate the compression ratio and then raise the block size appropriately (not necessarily to get it back to 64K).

      This should probably be added wherever the other HFile metrics are: RegionLoad.createRegions(..), and HServerLoad. HServerLoad is a Writable, so it may break serialization.

      Attachments

        1. 3927.txt
          7 kB
          Ted Yu
        2. 3927-v2.txt
          9 kB
          Ted Yu
        3. regionserver-showing-compression-ratio.png
          143 kB
          Ted Yu

        Activity

          People

            yuzhihong@gmail.com Ted Yu
            mcorgan Matt Corgan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: