Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7371

BKDReader could compress values better

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.2, 7.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      For compressing values, BKDReader only relies on shared prefixes in a block. We could probably easily do better. For instance there are only 256 possible values for the first byte of the dimension that the values are sorted by, yet we use a block size of 1024. So by using something simple like run-length compression we could save 6 bits per value on average.

        Attachments

        1. LUCENE-7371.patch
          23 kB
          Adrien Grand
        2. LUCENE-7371.patch
          22 kB
          Adrien Grand
        3. LUCENE-7371.patch
          17 kB
          Adrien Grand

          Activity

            People

            • Assignee:
              jpountz Adrien Grand
              Reporter:
              jpountz Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: