Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      After the big spans overhaul in LUCENE-6308, we can speed up SpanNearQuery a little more:

      • SpanNearQuery defaults to collectPayloads=true, but this requires a slower implementation, for an uncommon case. Use the faster no-payloads impl if the field doesn't actually have any payloads.
      • Use a simple array of Spans rather than List in NearSpans classes. This is iterated over often in the logic.

        Activity

        Hide
        Robert Muir added a comment -
        Report after iter 10:
        Chart saved to out.png... (wd: /home/rmuir/workspace/util/src/python)
                            Task   QPS trunk      StdDev   QPS patch      StdDev                Pct diff
                     MedSpanNear       75.69      (2.0%)       80.58      (3.9%)    6.5% (   0% -   12%)
                     LowSpanNear      233.30      (3.8%)      259.44      (6.5%)   11.2% (   0% -   22%)
                    HighSpanNear        9.43      (3.6%)       10.76      (7.5%)   14.0% (   2% -   25%)
        
        Show
        Robert Muir added a comment - Report after iter 10: Chart saved to out.png... (wd: /home/rmuir/workspace/util/src/python) Task QPS trunk StdDev QPS patch StdDev Pct diff MedSpanNear 75.69 (2.0%) 80.58 (3.9%) 6.5% ( 0% - 12%) LowSpanNear 233.30 (3.8%) 259.44 (6.5%) 11.2% ( 0% - 22%) HighSpanNear 9.43 (3.6%) 10.76 (7.5%) 14.0% ( 2% - 25%)
        Hide
        Paul Elschot added a comment -

        Oops, I thought by now ArrayList would be JIT-ed away, thanks.
        Also the UOE's in the NearSpansOrdered payload methods have gone in this patch, I had put these in to check the tests.

        Show
        Paul Elschot added a comment - Oops, I thought by now ArrayList would be JIT-ed away, thanks. Also the UOE's in the NearSpansOrdered payload methods have gone in this patch, I had put these in to check the tests.
        Hide
        Robert Muir added a comment -

        I removed the UOE because now the no-payload impl is used if a segment doesn't happen to have any payloads. But this is valid, the documents might just not have any.

        Show
        Robert Muir added a comment - I removed the UOE because now the no-payload impl is used if a segment doesn't happen to have any payloads. But this is valid, the documents might just not have any.
        Hide
        Adrien Grand added a comment -

        +1 too bad we can't expect ArrayList to always perform like a plain array

        Show
        Adrien Grand added a comment - +1 too bad we can't expect ArrayList to always perform like a plain array
        Hide
        ASF subversion and git services added a comment -

        Commit 1671078 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1671078 ]

        LUCENE-6388: Optimize SpanNearQuery

        Show
        ASF subversion and git services added a comment - Commit 1671078 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1671078 ] LUCENE-6388 : Optimize SpanNearQuery
        Hide
        ASF subversion and git services added a comment -

        Commit 1671081 from Robert Muir in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1671081 ]

        LUCENE-6388: Optimize SpanNearQuery

        Show
        ASF subversion and git services added a comment - Commit 1671081 from Robert Muir in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1671081 ] LUCENE-6388 : Optimize SpanNearQuery
        Hide
        Robert Muir added a comment -

        For now the check is implemented via Terms.getPayloads() until LUCENE-6390 is fixed.

        Show
        Robert Muir added a comment - For now the check is implemented via Terms.getPayloads() until LUCENE-6390 is fixed.
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development