Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10231

Cursor value always different for last page with sorting by a date based function using NOW

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Information Provided
    • Affects Version/s: 4.10.2
    • Fix Version/s: 6.5
    • Labels:
      None

      Description

      Cursor based results fetching is a deal breaker for search performance.
      It works extremely well when paging using sort by field(s).

      Example, that works (Id is unique field in the schema):
      Query:

      http://solr-host:8983/solr/documents/select?q=*:*&fq=DocumentId:76581059&cursorMark=AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==&fl=DocumentId&sort=UserId+asc%2CId+desc&rows=1
      

      Response:

      <response>
      <lst name="responseHeader">
      <int name="status">0</int>
      <int name="QTime">4</int>
      <lst name="params">
      <str name="q">*:*</str>
      <str name="fl">DocumentId</str>
      <str name="cursorMark">AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==</str>
      <str name="fq">DocumentId:76581059</str>
      <str name="sort">UserId asc,Id desc</str>
      <str name="rows">1</str>
      </lst>
      </lst>
      <result name="response" numFound="1" start="0"/>
      <str name="nextCursorMark">AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==</str>
      </response>
      

      nextCursorMark equals to cursorMark and so we know this is last page.

      However, sorting by function behaves differently:
      Query:

      http://solr-host:8983/solr/documents/select?rows=1&q=*:*&fq=DocumentId:76581059&cursorMark=AoIFQf9yCCAAAAAuU1NSLTc2NTgxMDU5LTE=&fl=DocumentId&sort=min(ms(NOW,DynamicDateField_1),ms(NOW,DynamicDateField_12),ms(NOW,DynamicDateField_3),ms(NOW,DynamicDateField_5))%20asc,Id%20desc
      

      Response:

      <response>
      <lst name="responseHeader">
      <int name="status">0</int>
      <int name="QTime">6</int>
      <lst name="params">
      <str name="q">*:*</str>
      <str name="fl">DocumentId</str>
      <str name="cursorMark">AoIFQf9yCCAAAAAuU1NSLTc2NTgxMDU5LTE=</str>
      <str name="fq">DocumentId:76581059</str>
      <str name="sort">
      min(ms(NOW,DynamicDateField_1),ms(NOW,DynamicDateField_12),ms(NOW,DynamicDateField_3),ms(NOW,DynamicDateField_5)) asc,Id desc
      </str>
      <str name="rows">1</str>
      </lst>
      </lst>
      <result name="response" numFound="1" start="0">
      <doc>
      <str name="DocumentId">76581059</str>
      </doc>
      </result>
      <str name="nextCursorMark">AoIFQf9yFyAAAAAuU1NSLTc2NTgxMDU5LTE=</str>
      </response>
      

      nextCursorMark does not equal to cursorMark, which suggests there are more results. Which is not true (numFound=1). And so the client goes into infinite loop.

        Attachments

          Activity

            People

            • Assignee:
              hossman Chris M. Hostetter
              Reporter:
              dmitry_key Dmitry Kan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: