[SOLR-15290] Better Docs/Tests/Warnings/Defaults for SignatureUpdateProcessorFactory in SolrCloud

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Chris M. Hostetter
• Created: 25/Mar/21 18:22
• Updated: 12/Apr/21 20:18

Description

I was recently helping someone who had very weird symptoms of replicas getting out of sync, that turned out to be because of using SignatureUpdateProcessorFactory to "de-duplicate" documents that had different unique keys, but identical computed "signatures".

Although they had customized the fields configuration of SignatureUpdateProcessorFactory, most of the (bad) behavior came from the defaults:

      overwriteDupes = params.getBool("overwriteDupes", true);
      signatureField = params.get("signatureField", "signatureField");

...in spite of the fact that this combination does not – and has never – worked with SolrCloud: SOLR-3473

I'm opening this issue to serve as a Parent for a few Sub-Tasks. Some of which I hope to takle imminently, and some of which are just ideas for the future.

Issue Links

• relates to: SOLR-3473 Distributed deduplication broken when using non-uniqueKey for signatureField

Sub-Tasks

• Deprecate/remove overwriteDupes option from SignatureUpdateProcessorFactory: Open
• Support "post-indexing" cleanup of documents with duplicate signatures: Open