Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-26405

IntegrationTestLoadSmallValues

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Minor
    • Resolution: Later
    • None
    • None
    • integration tests, test
    • None

    Description

      This integration test emulates a use case that stores a lot of small values into a table that would likely be heavily indexed (ROW_INDEX_V1, small blocks, etc.), an application that crowdsources weather (temperature) observation data. This IT can be used to test and optimize compression settings for such cases. It comes with a companion utility, HFileBlockExtracter, which extracts block data from HFiles into a set of individual files for each block's data, for use in training external compression dictionaries, perhaps with ZStandard's 'zstd' utility.

      See javadoc on class IntegrationTestLoadSmallValues for usage.

      This was used to test the changes on HBASE-26353.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue HBASE-26353 Support loadable dictionaries in hbase-compression-zstd

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link GitHub Pull Request #3802

          Activity

            People

              Unassigned Unassigned
              apurtell Andrew Kyle Purtell
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: