Lucene - Core
  1. Lucene - Core
  2. LUCENE-6569

MultiFunction.anyExists - creating FunctionValues[] objects for every document

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Lucene Fields:
      New
    • Flags:
      Patch

      Description

      In the class org.apache.lucene.queries.function.valuesource.MultiFunction there is the following method signature (line 52)

      public static boolean allExists(int doc, FunctionValues... values)

      this method is called from the class org.apache.lucene.queries.function.valuesource.DualFloatFunction (line 68)

      public boolean exists(int doc)

      { return MultiFunction.allExists(doc, aVals, bVals); }

      Because MultiFunction.allExists uses Java varargs syntax ("...") a new FunctionValues[] object will be created every time this call takes place.

      The problem is that the call takes place in a document level function, which means that it will create new objects in the heap for every document in the query results.

      for example if you use the following boost function (where ds and dc1 are both TrieDateField)

      bf=min(ms(ds,dc1),604800000)

      You will get extra objects created for each document in the result set, which has a big impact on performance and memory usage if you are searching a large result set.

      1. SOLR-7618.patch
        7 kB
        Hoss Man
      2. SOLR-7618.patch
        3 kB
        Jacob Graves

        Activity

        Hide
        Xu Zhang added a comment -

        It would be really nice if you upload a patch here. http://wiki.apache.org/solr/HowToContribute

        Show
        Xu Zhang added a comment - It would be really nice if you upload a patch here. http://wiki.apache.org/solr/HowToContribute
        Hide
        Jacob Graves added a comment -

        added the patch.

        Show
        Jacob Graves added a comment - added the patch.
        Hide
        Hoss Man added a comment -

        Great point Jacob.

        A few updates to your patch...

        • fix formatting (we use 2 spaces for indents)
        • improved javadocs & linkages
        • added equivilence testing between each method version (to future proof against someone changing one of these methods in the future and making them no longer behave consistently)
        Show
        Hoss Man added a comment - Great point Jacob. A few updates to your patch... fix formatting (we use 2 spaces for indents) improved javadocs & linkages added equivilence testing between each method version (to future proof against someone changing one of these methods in the future and making them no longer behave consistently)
        Hide
        ASF subversion and git services added a comment -

        Commit 1685687 from hossman@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1685687 ]

        LUCENE-6569: Optimize MultiFunction.anyExists and allExists to eliminate excessive array creation in common 2 argument usage

        Show
        ASF subversion and git services added a comment - Commit 1685687 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1685687 ] LUCENE-6569 : Optimize MultiFunction.anyExists and allExists to eliminate excessive array creation in common 2 argument usage
        Hide
        ASF subversion and git services added a comment -

        Commit 1685689 from hossman@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1685689 ]

        LUCENE-6569: Optimize MultiFunction.anyExists and allExists to eliminate excessive array creation in common 2 argument usage (merge r1685687)

        Show
        ASF subversion and git services added a comment - Commit 1685689 from hossman@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1685689 ] LUCENE-6569 : Optimize MultiFunction.anyExists and allExists to eliminate excessive array creation in common 2 argument usage (merge r1685687)
        Hide
        Hoss Man added a comment -

        Thanks!

        Show
        Hoss Man added a comment - Thanks!
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Hoss Man
            Reporter:
            Jacob Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 2h
              2h
              Remaining:
              Remaining Estimate - 2h
              2h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development