Lucene - Core
  1. Lucene - Core
  2. LUCENE-3892

Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta, Simple9/16/64, etc.)

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0, 5.0
    • Component/s: None
    • Lucene Fields:
      New

      Description

      On the flex branch we explored a number of possible intblock
      encodings, but for whatever reason never brought them to completion.
      There are still a number of issues opened with patches in different
      states.

      Initial results (based on prototype) were excellent (see
      http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
      ).

      I think this would make a good GSoC project.

      1. LUCENE-3892-trunk.patch
        2.75 MB
        Michael McCandless
      2. LUCENE-3892-pfor-compress-slow-estimate.patch
        36 kB
        Han Jiang
      3. LUCENE-3892-pfor-compress-iterate-numbits.patch
        31 kB
        Han Jiang
      4. LUCENE-3892-non-specialized.patch
        7 kB
        Michael McCandless
      5. LUCENE-3892-liveregs.patch
        4 kB
        Michael McCandless
      6. LUCENE-3892-javadocs.patch
        19 kB
        Michael McCandless
      7. LUCENE-3892-handle_open_files.patch
        2 kB
        Han Jiang
      8. LUCENE-3892-for&pfor-with-javadoc.patch
        29 kB
        Han Jiang
      9. LUCENE-3892-direct-IntBuffer.patch
        148 kB
        Michael McCandless
      10. LUCENE-3892-bulkVInt.patch
        5 kB
        Michael McCandless
      11. LUCENE-3892-BlockTermScorer.patch
        10 kB
        Michael McCandless
      12. LUCENE-3892-blockpfor.patch
        15 kB
        Han Jiang
      13. LUCENE-3892-blockFor-with-packedints-decoder.patch
        27 kB
        Han Jiang
      14. LUCENE-3892-blockFor-with-packedints-decoder.patch
        27 kB
        Han Jiang
      15. LUCENE-3892-blockFor-with-packedints.patch
        17 kB
        Han Jiang
      16. LUCENE-3892-blockFor&packedecoder(comp).patch
        76 kB
        Han Jiang
      17. LUCENE-3892-blockFor&hardcode(base).patch
        83 kB
        Han Jiang
      18. LUCENE-3892_settings.patch
        6 kB
        Robert Muir
      19. LUCENE-3892_settings.patch
        9 kB
        Robert Muir
      20. LUCENE-3892_pulsing_support.patch
        12 kB
        Han Jiang
      21. LUCENE-3892_pfor_unfold_method.patch
        148 kB
        Han Jiang
      22. LUCENE-3892_for_unfold_method.patch
        139 kB
        Han Jiang
      23. LUCENE-3892_for_int[].patch
        89 kB
        Han Jiang
      24. LUCENE-3892_for_byte[].patch
        131 kB
        Han Jiang

        Issue Links

          Activity

          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4283 [ LUCENE-4283 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4283 [ LUCENE-4283 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4239 [ LUCENE-4239 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4239 [ LUCENE-4239 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4161 [ LUCENE-4161 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4161 [ LUCENE-4161 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4225 [ LUCENE-4225 ]
          Gavin made changes -
          Link This issue relates to LUCENE-4225 [ LUCENE-4225 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-liveregs.patch [ 12542291 ]
          Michael McCandless made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 5.0 [ 12321663 ]
          Fix Version/s 4.0 [ 12322550 ]
          Fix Version/s 4.1 [ 12321140 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-trunk.patch [ 12541482 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-javadocs.patch [ 12541479 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-blockpfor.patch [ 12540045 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-non-specialized.patch [ 12539952 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-bulkVInt.patch [ 12539881 ]
          Han Jiang made changes -
          Link This issue is related too LUCENE-4283 [ LUCENE-4283 ]
          Han Jiang made changes -
          Link This issue is related too LUCENE-4283 [ LUCENE-4283 ]
          Han Jiang made changes -
          Link This issue relates LUCENE-4283 [ LUCENE-4283 ]
          Han Jiang made changes -
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536010 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536235 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536391 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor.patch [ 12534297 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_for.patch [ 12532413 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12530642 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12530703 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12532412 ]
          Han Jiang made changes -
          Link This issue relates LUCENE-4239 [ LUCENE-4239 ]
          Han Jiang made changes -
          Link This issue relates LUCENE-4161 [ LUCENE-4161 ]
          Han Jiang made changes -
          Link This issue relates LUCENE-4225 [ LUCENE-4225 ]
          Han Jiang made changes -
          Han Jiang made changes -
          Han Jiang made changes -
          Han Jiang made changes -
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536427 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536431 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536427 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536420 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536420 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536391 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536235 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor-with-javadoc.patch [ 12536010 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pulsing_support.patch [ 12535699 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pulsing_support.patch [ 12535700 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pulsing_support.patch [ 12535699 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-handle_open_files.patch [ 12535698 ]
          Han Jiang made changes -
          Attachment LUCENE-3892-for&pfor.patch [ 12534297 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-BlockTermScorer.patch [ 12533086 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_for_byte[].patch [ 12532905 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_for_int[].patch [ 12532888 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor_unfold_method.patch [ 12532690 ]
          Attachment LUCENE-3892_for_unfold_method.patch [ 12532691 ]
          Michael McCandless made changes -
          Attachment LUCENE-3892-direct-IntBuffer.patch [ 12532679 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12532408 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_for.patch [ 12532409 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12532412 ]
          Attachment LUCENE-3892_for.patch [ 12532413 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12532408 ]
          Attachment LUCENE-3892_for.patch [ 12532409 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12530703 ]
          Han Jiang made changes -
          Attachment LUCENE-3892_pfor.patch [ 12530642 ]
          Robert Muir made changes -
          Fix Version/s 4.1 [ 12321140 ]
          Fix Version/s 4.0 [ 12314025 ]
          Dawid Weiss made changes -
          Link This issue incorporates LUCENE-3987 [ LUCENE-3987 ]
          Robert Muir made changes -
          Attachment LUCENE-3892_settings.patch [ 12524472 ]
          Robert Muir made changes -
          Field Original Value New Value
          Attachment LUCENE-3892_settings.patch [ 12524461 ]
          Michael McCandless created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Michael McCandless
            • Votes:
              1 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development