Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-274

[PATCH] to store binary fields with compression

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core/index
    • Labels:
      None
    • Environment:

      Operating System: All
      Platform: All

    • Bugzilla Id:
      31149

      Description

      hi all,

      as promised here is the enhancement for the binary field patch with optional
      compression. The attachment includes all necessary diffs based on the latest
      version from CVS. There is also a small junit test case to test the core
      functionality for binary field compression. The base implementation for binary
      fields where this patch relies on, can be found in patch #29370. The existing
      unit tests pass fine.

      For testing binary fields and compression, I'm creating an index from 2700 plain
      text files (avg. 6kb per file) and store all file content within that index
      without using compression. The test was created using the IndexFiles class from
      the demo distribution. Setting up the index and storing all content without
      compression took about 60 secs and the final index size was 21 MB. Running the
      same test, switching compression on, the time to index increase to 75 secs, but
      the final index size shrinks to 13 MB. This is less than the plain text files
      them self need in the file system (15 MB)

      Hopefully this patch helps people dealing with huge index and want to store more
      than just 300 bytes per document to display a well formed summary.

      regards
      Bernhard

        Attachments

        1. ASF.LICENSE.NOT.GRANTED--compress-patch.tgz
          6 kB
          Bernhard Messer
        2. ASF.LICENSE.NOT.GRANTED--compressPatch.tgz
          5 kB
          Bernhard Messer

          Activity

            People

            • Assignee:
              java-dev@lucene.apache.org Lucene Developers
              Reporter:
              bernhard.messer@intrafind.de Bernhard Messer
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: