Lucene - Core
  1. Lucene - Core
  2. LUCENE-6133

improve default stored fields merge algorithm

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.0, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      This is not very efficient today.

      We should try to improve the default one and not rely so much upon bulk merges. At the least we can use a specialized visitor, save some cpu, and avoid creating so much garbage.

      Another pain point in default merge is all the string creation and unicode encode/decoding back and forth. I don't tackle that here... but that would be fantastic to fix in another issue.

        Activity

        Hide
        Adrien Grand added a comment -

        +1 I'm wondering if we should bring the reuse of the byte[] to DataOutput in general?

        Show
        Adrien Grand added a comment - +1 I'm wondering if we should bring the reuse of the byte[] to DataOutput in general?
        Hide
        Robert Muir added a comment -

        Certainly not. If you want things to be fast, dont use string!!!!!!

        Show
        Robert Muir added a comment - Certainly not. If you want things to be fast, dont use string!!!!!!
        Hide
        ASF subversion and git services added a comment -

        Commit 1648327 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1648327 ]

        LUCENE-6133: improve default stored fields merge algorithm

        Show
        ASF subversion and git services added a comment - Commit 1648327 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1648327 ] LUCENE-6133 : improve default stored fields merge algorithm
        Hide
        ASF subversion and git services added a comment -

        Commit 1648332 from Robert Muir in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1648332 ]

        LUCENE-6133: improve default stored fields merge algorithm

        Show
        ASF subversion and git services added a comment - Commit 1648332 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1648332 ] LUCENE-6133 : improve default stored fields merge algorithm
        Hide
        Anshum Gupta added a comment -

        Bulk close after 5.0 release.

        Show
        Anshum Gupta added a comment - Bulk close after 5.0 release.

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development