[SOLR-12363] Duplicates with random search, cursors, and fixed seed

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 5.3.1
• Assignee: Unassigned
• Reporter: Alexander S.
• Created: 16/May/18 11:52
• Updated: 08/Jun/19 15:15

Description

We do have a SolrCloud cluster and just updated one of our views to use cursors with the random order. Our goal was to use an infinite scroll with the random ordering so we can shuffle results once every 24 hours.

To do so we save the seed that we use in our random order to the cookies with the 24 hours expiration period, which didn't work as expected:

1. Results are shuffled with every request (every time we pass the initial cursor value "*" and the same random value for ordering we already used).
2. Results contain duplicates sometimes. Not a lot of them, but from time to time they appear.

In our schema.xml we have:

<fieldType name="rand" class="solr.RandomSortField" omitNorms="true"/>
<dynamicField name="random_*" stored="false" type="rand" multiValued="false" indexed="true"/>

In our search requests, we order by random_123 asc, id asc, where 123 is the seed from cookies.

Here is the page https://awards.wegohealth.com/nominees

Even when I try to get the "next page" URL from google chrome developer console and open it in separate tabs it yields different results: https://awards.wegohealth.com/nominees?cursor=AoJYmYbyATRBd2FyZDo6Tm9taW5lZSAxMzI0Mg%3D%3D

So it feels like the seed parameter we use is ignored or every shard understands it differently, not sure.

On the screenshots, you can see the URL is the same and results are different.