Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.0, 5.1, 5.2, 5.2.1
    • Fix Version/s: 5.3
    • Component/s: None
    • Labels:
      None

      Description

      SolrQueryTimeoutImpl can be slow for some use cases, for example, in cases with many terms, where shouldExit() is called many times.
      See http://search-lucene.com/m/l6pAi1HLrodLhNUd

      1. SOLR-7875.patch
        2 kB
        Tomás Fernández Löbbe

        Activity

        Hide
        Tomás Fernández Löbbe added a comment -

        If we leave the unset value as null (instead of nanoTime() + nanoTime() + Long.MAX_VALUE) and we do a null check in the shouldExit() method there is already a big improvement. Still not as good as without the ExitableDirectoryReader wrapper:

          Solr-5_3_0-Not-Exitable-1 Solr-5_3_0-Not-Exitable-2 Solr-5_3_0-Exitable-1 Solr-5_3_0-Exitable-2 Solr-5_3_0-branch5-1 Solr-5_3_0-branch5-2
        Average 395 393 442 440 2603 2595
        p10 95 90 105 103 558 551
        p50 380 378 426 423 2520 2507
        p75 581 579 646 645 3839 3839
        p90 707 706 789 788 4710 4698
        p95 782 778 870 870 5234 5232
        p99 989 984 1104 1105 6671 6700

        This is with a test that runs 3k boolean queries with filters on double fields (more details in the email thread linked in the description of the ticket).

        • Solr-5_3_0-Not-Exitable-# is a snapshot of 5x branch that completely skips the ExitableDirectoryReader wrapping
        • Solr-5_3_0-Exitable-# is the same snapshot but with the patch that does the null checks.
        • Solr-5_3_0-branch5-# is the same snapshot but without any changes.
        Show
        Tomás Fernández Löbbe added a comment - If we leave the unset value as null (instead of nanoTime() + nanoTime() + Long.MAX_VALUE) and we do a null check in the shouldExit() method there is already a big improvement. Still not as good as without the ExitableDirectoryReader wrapper:   Solr-5_3_0-Not-Exitable-1 Solr-5_3_0-Not-Exitable-2 Solr-5_3_0-Exitable-1 Solr-5_3_0-Exitable-2 Solr-5_3_0-branch5-1 Solr-5_3_0-branch5-2 Average 395 393 442 440 2603 2595 p10 95 90 105 103 558 551 p50 380 378 426 423 2520 2507 p75 581 579 646 645 3839 3839 p90 707 706 789 788 4710 4698 p95 782 778 870 870 5234 5232 p99 989 984 1104 1105 6671 6700 This is with a test that runs 3k boolean queries with filters on double fields (more details in the email thread linked in the description of the ticket). Solr-5_3_0-Not-Exitable-# is a snapshot of 5x branch that completely skips the ExitableDirectoryReader wrapping Solr-5_3_0-Exitable-# is the same snapshot but with the patch that does the null checks. Solr-5_3_0-branch5-# is the same snapshot but without any changes.
        Hide
        Anshum Gupta added a comment -

        LGTM. This is a low hanging change that gets us a good improvement. I think it makes sense for this to go into 5.3.

        P.S.: Also, the only way to work without the wrapping would involve maintaining 2 readers in SolrIndexSearcher.

        Show
        Anshum Gupta added a comment - LGTM. This is a low hanging change that gets us a good improvement. I think it makes sense for this to go into 5.3. P.S.: Also, the only way to work without the wrapping would involve maintaining 2 readers in SolrIndexSearcher.
        Hide
        Tomás Fernández Löbbe added a comment -

        thanks for the review. I'll commit this and merge to the 5.3 branch.

        Also, the only way to work without the wrapping would involve maintaining 2 readers in SolrIndexSearcher.

        Let's discuss that in SOLR-7876

        Show
        Tomás Fernández Löbbe added a comment - thanks for the review. I'll commit this and merge to the 5.3 branch. Also, the only way to work without the wrapping would involve maintaining 2 readers in SolrIndexSearcher. Let's discuss that in SOLR-7876
        Hide
        Steve Rowe added a comment -

        +1, LGTM, thanks Tomás.

        Show
        Steve Rowe added a comment - +1, LGTM, thanks Tomás.
        Hide
        ASF subversion and git services added a comment -

        Commit 1694574 from Tomás Fernández Löbbe in branch 'dev/trunk'
        [ https://svn.apache.org/r1694574 ]

        SOLR-7875: Speedup SolrQueryTimeoutImpl

        Show
        ASF subversion and git services added a comment - Commit 1694574 from Tomás Fernández Löbbe in branch 'dev/trunk' [ https://svn.apache.org/r1694574 ] SOLR-7875 : Speedup SolrQueryTimeoutImpl
        Hide
        ASF subversion and git services added a comment -

        Commit 1694575 from Tomás Fernández Löbbe in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1694575 ]

        SOLR-7875: Speedup SolrQueryTimeoutImpl

        Show
        ASF subversion and git services added a comment - Commit 1694575 from Tomás Fernández Löbbe in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694575 ] SOLR-7875 : Speedup SolrQueryTimeoutImpl
        Hide
        ASF subversion and git services added a comment -

        Commit 1694576 from Tomás Fernández Löbbe in branch 'dev/branches/lucene_solr_5_3'
        [ https://svn.apache.org/r1694576 ]

        SOLR-7875: Speedup SolrQueryTimeoutImpl

        Show
        ASF subversion and git services added a comment - Commit 1694576 from Tomás Fernández Löbbe in branch 'dev/branches/lucene_solr_5_3' [ https://svn.apache.org/r1694576 ] SOLR-7875 : Speedup SolrQueryTimeoutImpl
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Tomás Fernández Löbbe
            Reporter:
            Tomás Fernández Löbbe
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development