Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9, 3.1, 4.0-ALPHA
    • Component/s: modules/benchmark
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      bzip compression can aid the benchmark package by not requiring extracting bzip files (such as enwiki) in order to index them. The plan is to add a config parameter bzip.compression=true/false and in the relevant tasks either decompress the input file or compress the output file using the bzip streams.
      It will add a dependency on ant.jar which contains two classes similar to GZIPOutputStream and GZIPInputStream which compress/decompress files using the bzip algorithm.

      bzip is known to be superior in its compression performance to the gzip algorithm (~20% better compression), although it does the compression/decompression a bit slower.

      I wil post a patch which adds this parameter and implement it in LineDocMaker, EnwikiDocMaker and WriteLineDoc task. Maybe even add the capability to DocMaker or some of the super classes, so it can be inherited by all sub-classes.

      1. LUCENE-1591.patch
        2 kB
        Mark Miller
      2. LUCENE-1591.patch
        47 kB
        Shai Erera
      3. LUCENE-1591.patch
        47 kB
        Shai Erera
      4. LUCENE-1591.patch
        45 kB
        Shai Erera
      5. LUCENE-1591.patch
        35 kB
        Shai Erera
      6. commons-compress-dev20090413.jar
        137 kB
        Uwe Schindler
      7. LUCENE-1591.patch
        21 kB
        Shai Erera
      8. commons-compress-dev20090413.jar
        137 kB
        Uwe Schindler
      9. LUCENE-1591.patch
        20 kB
        Shai Erera
      10. LUCENE-1591.patch
        15 kB
        Shai Erera

        Activity

        Shai Erera created issue -
        Shai Erera made changes -
        Field Original Value New Value
        Attachment LUCENE-1591.patch [ 12405251 ]
        Attachment ant-1.7.1.jar [ 12405252 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405284 ]
        Uwe Schindler made changes -
        Attachment commons-compress-dev20090413.jar [ 12405309 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405311 ]
        Uwe Schindler made changes -
        Attachment commons-compress-dev20090413.jar [ 12405336 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405354 ]
        Shai Erera made changes -
        Attachment ant-1.7.1.jar [ 12405252 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405435 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405535 ]
        Shai Erera made changes -
        Attachment LUCENE-1591.patch [ 12405611 ]
        Michael McCandless made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Mark Miller made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Lucene Fields [New] [New, Patch Available]
        Assignee Mark Miller [ markrmiller@gmail.com ]
        Mark Miller made changes -
        Attachment LUCENE-1591.patch [ 12412638 ]
        Mark Miller made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Mark Miller made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Michael McCandless made changes -
        Resolution Fixed [ 1 ]
        Status Closed [ 6 ] Reopened [ 4 ]
        Michael McCandless made changes -
        Fix Version/s 3.1 [ 12314822 ]
        Fix Version/s 4.0 [ 12314025 ]
        Michael McCandless made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Mark Thomas made changes -
        Workflow jira [ 12460280 ] Default workflow, editable Closed status [ 12563320 ]
        Mark Thomas made changes -
        Workflow Default workflow, editable Closed status [ 12563320 ] jira [ 12584437 ]
        Grant Ingersoll made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Mark Miller
            Reporter:
            Shai Erera
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development