Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-14413

allow timeAllowed and cursorMark parameters

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 8.8, main (9.0)
    • Component/s: search
    • Labels:
      None

      Description

      Ever since cursorMarks were introduced in SOLR-5463 in 2014, cursorMark and timeAllowed parameters were not allowed in combination ("Can not search using both cursorMark and timeAllowed")

      , from QueryComponent.java]:

       

       
       if (null != rb.getCursorMark() && 0 < timeAllowed) {
        // fundamentally incompatible
        throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, "Can not search using both " + CursorMarkParams.CURSOR_MARK_PARAM + " and " + CommonParams.TIME_ALLOWED);
      } 

      While theoretically impure to use them in combination, it is often desirable to support cursormarks-style deep paging and attempt to protect Solr nodes from runaway queries using timeAllowed, in the hopes that most of the time, the query completes in the allotted time, and there is no conflict.

       

      However if the query takes too long, it may be preferable to end the query and protect the Solr node and provide the user with a somewhat inaccurate sorted list. As noted in SOLR-6930, SOLR-5986 and others, timeAllowed is frequently used to prevent runaway load.  In fact, cursorMark and shards.tolerant are allowed in combination, so any argument in favor of purity would be a bit muddied in my opinion.

       

      This was discussed once in the mailing list that I can find: https://mail-archives.apache.org/mod_mbox/lucene-solr-user/201506.mbox/%3C5591740B.4080807@elyograg.org%3E It did not look like there was strong support for preventing the combination.

       

      I have tested cursorMark and timeAllowed combination together, and even when partial results are returned because the timeAllowed is exceeded, the cursorMark response value is still valid and reasonable.

        Attachments

        1. image-2020-08-18-16-56-41-736.png
          381 kB
          John Gallagher
        2. image-2020-08-18-16-56-59-178.png
          137 kB
          John Gallagher
        3. image-2020-08-21-14-18-36-229.png
          177 kB
          John Gallagher
        4. Screen Shot 2020-10-23 at 10.08.26 PM.png
          214 kB
          John Gallagher
        5. Screen Shot 2020-10-23 at 10.09.11 PM.png
          239 kB
          John Gallagher
        6. SOLR-14413.patch
          3 kB
          John Gallagher
        7. SOLR-14413.testfix.patch
          0.6 kB
          John Gallagher
        8. SOLR-14413-bram.patch
          10 kB
          Bram Van Dam
        9. SOLR-14413-jg-update1.patch
          10 kB
          John Gallagher
        10. SOLR-14413-jg-update2.patch
          13 kB
          John Gallagher
        11. SOLR-14413-jg-update3.patch
          14 kB
          John Gallagher
        12. timeallowed_cursormarks_results.txt
          8 kB
          John Gallagher

          Issue Links

            Activity

              People

              • Assignee:
                mdrob Mike Drob
                Reporter:
                slackhappy John Gallagher
              • Votes:
                2 Vote for this issue
                Watchers:
                8 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h 10m
                  3h 10m