Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      A burst of "expensive" queries can cause contention within lucene, greatly reducing effective throughput and causing more and more queries to stack up.

        Issue Links

          Activity

          Yonik Seeley created issue -
          Hide
          Yonik Seeley added a comment -

          One method to "fix" this would be to limit the number of queries that may be executing at any one time (queueing).
          http://www.nabble.com/TermInfosReader-lazy-term-index-reading-tf3162657.html#a8776274

          One could do this in the servlet container by limiting the number of threads in the thread pool, but would that lock out the admin pages?

          A more flexible way might be to put the queueing in the SolrCore.exec() method.
          Admin pages could bypass the exec() method by calling a handler directly, or could set a priority on the request that avoided queueing.

          Show
          Yonik Seeley added a comment - One method to "fix" this would be to limit the number of queries that may be executing at any one time (queueing). http://www.nabble.com/TermInfosReader-lazy-term-index-reading-tf3162657.html#a8776274 One could do this in the servlet container by limiting the number of threads in the thread pool, but would that lock out the admin pages? A more flexible way might be to put the queueing in the SolrCore.exec() method. Admin pages could bypass the exec() method by calling a handler directly, or could set a priority on the request that avoided queueing.
          Hide
          Yonik Seeley added a comment -

          I'm starting to like this idea...
          Before, if some queries were taking a long time, you couldn't tell which ones because they don't get logged until after completion.

          But if you are queuing in a manner that lets you keep track of currently executing requests, you could export that info to the admin page, along with how long each query has been running.

          Show
          Yonik Seeley added a comment - I'm starting to like this idea... Before, if some queries were taking a long time, you couldn't tell which ones because they don't get logged until after completion. But if you are queuing in a manner that lets you keep track of currently executing requests, you could export that info to the admin page, along with how long each query has been running.
          Hide
          Lance Norskog added a comment -

          Our use case is the classic battle between interactive use and running reports: reports use long queries that shut out interactive use.

          One option is to have two search servlets instead of one, where each can have a separate queueing depth. One servlet is reserved for long queries and may only have one or two queries active at one time, while the other servlet is for smaller interactive queries and might allow 5 concurrent queries.

          Show
          Lance Norskog added a comment - Our use case is the classic battle between interactive use and running reports: reports use long queries that shut out interactive use. One option is to have two search servlets instead of one, where each can have a separate queueing depth. One servlet is reserved for long queries and may only have one or two queries active at one time, while the other servlet is for smaller interactive queries and might allow 5 concurrent queries.
          Hide
          Ian Holsman added a comment - - edited

          the 2nd option is to use a query timeout like the one in #SOLR-502 so that "interactive" queries can have a fixed end time which kill the badly behaved queries

          Show
          Ian Holsman added a comment - - edited the 2nd option is to use a query timeout like the one in # SOLR-502 so that "interactive" queries can have a fixed end time which kill the badly behaved queries
          Jed Wesley-Smith made changes -
          Field Original Value New Value
          Link This issue depends on LUCENE-1609 [ LUCENE-1609 ]
          Hide
          Erick Erickson added a comment -

          Cleaning up old JIRAs, re-open if necessary.

          Show
          Erick Erickson added a comment - Cleaning up old JIRAs, re-open if necessary.
          Erick Erickson made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Won't Fix [ 2 ]
          Gavin made changes -
          Link This issue depends on LUCENE-1609 [ LUCENE-1609 ]
          Gavin made changes -
          Link This issue depends upon LUCENE-1609 [ LUCENE-1609 ]

            People

            • Assignee:
              Unassigned
              Reporter:
              Yonik Seeley
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development