[SOLR-10231] Cursor value always different for last page with sorting by a date based function using NOW

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Information Provided
• Affects Version/s: 4.10.2
• Fix Version/s: 6.5
• Component/s: SearchComponents - other
• Assignee: Chris M. Hostetter
• Reporter: Dmitry Kan
• Created: 06/Mar/17 07:23
• Updated: 08/Jun/19 15:30
• Resolved: 14/Mar/17 00:36

Description

Cursor based results fetching is a deal breaker for search performance.
It works extremely well when paging using sort by field(s).

Example, that works (Id is unique field in the schema):
Query:

http://solr-host:8983/solr/documents/select?q=*:*&fq=DocumentId:76581059&cursorMark=AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==&fl=DocumentId&sort=UserId+asc%2CId+desc&rows=1

Response:

<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">4</int>
<lst name="params">
<str name="q">*:*</str>
<str name="fl">DocumentId</str>
<str name="cursorMark">AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==</str>
<str name="fq">DocumentId:76581059</str>
<str name="sort">UserId asc,Id desc</str>
<str name="rows">1</str>
</lst>
</lst>
<result name="response" numFound="1" start="0"/>
<str name="nextCursorMark">AoIGAAAAAC5TU1ItNzY1ODEwNTktMQ==</str>
</response>

nextCursorMark equals to cursorMark and so we know this is last page.

However, sorting by function behaves differently:
Query:

http://solr-host:8983/solr/documents/select?rows=1&q=*:*&fq=DocumentId:76581059&cursorMark=AoIFQf9yCCAAAAAuU1NSLTc2NTgxMDU5LTE=&fl=DocumentId&sort=min(ms(NOW,DynamicDateField_1),ms(NOW,DynamicDateField_12),ms(NOW,DynamicDateField_3),ms(NOW,DynamicDateField_5))%20asc,Id%20desc

Response:

<response>
<lst name="responseHeader">
<int name="status">0</int>
<int name="QTime">6</int>
<lst name="params">
<str name="q">*:*</str>
<str name="fl">DocumentId</str>
<str name="cursorMark">AoIFQf9yCCAAAAAuU1NSLTc2NTgxMDU5LTE=</str>
<str name="fq">DocumentId:76581059</str>
<str name="sort">
min(ms(NOW,DynamicDateField_1),ms(NOW,DynamicDateField_12),ms(NOW,DynamicDateField_3),ms(NOW,DynamicDateField_5)) asc,Id desc
</str>
<str name="rows">1</str>
</lst>
</lst>
<result name="response" numFound="1" start="0">
<doc>
<str name="DocumentId">76581059</str>
</doc>
</result>
<str name="nextCursorMark">AoIFQf9yFyAAAAAuU1NSLTc2NTgxMDU5LTE=</str>
</response>

nextCursorMark does not equal to cursorMark, which suggests there are more results. Which is not true (numFound=1). And so the client goes into infinite loop.