[LUCENE-8264] Allow an option to rewrite all segments

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Lucene Fields: New
• Assignee: Erick Erickson
• Reporter: Erick Erickson
• Created: 22/Apr/18 03:33
• Updated: 28/Aug/22 15:28
• Resolved: 18/Aug/18 03:35

Description

For the background, see SOLR-12259.

There are several use-cases that would be much easier, especially during upgrades, if we could specify that all segments get rewritten.

One example: Upgrading 5x->6x->7x. When segments are merged, they're rewritten into the current format. However, there's no guarantee that a particular segment ever gets merged so the 6x-7x upgrade won't necessarily be successful.

How many merge policies support this is an open question. I propose to start with TMP and raise other JIRAs as necessary for other merge policies.

So far the usual response has been "re-index from scratch", but that's increasingly difficult as systems get larger.

Issue Links

• is related to: LUCENE-8004 IndexUpgraderTool should rewrite segments rather than forceMerge
• relates to: LUCENE-8004 IndexUpgraderTool should rewrite segments rather than forceMerge
• relates to: SOLR-12259 Robustly upgrade indexes