[LUCENE-9669] N-2 compatibility for file formats

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 9.0
• Lucene Fields: New
• Assignee: Simon Willnauer
• Reporter: Simon Willnauer
• Created: 14/Jan/21 15:51
• Updated: 28/Aug/22 16:12
• Resolved: 22/Jan/21 08:56

Description

Currently Lucene supports reading and writing indices that have been
created with the current or previous (N-1) version of Lucene. Lucene
refuses to open an index created by N-2 or earlier versions.
I would like to propose that Lucene adds support for opening indices
created by version N-2 in read-only mode. Here's what I have in mind:

• Read-only support. You can open a reader on an index created by
  version N-2, but you cannot open an IndexWriter on it, meaning that
  you can neither delete, update, add documents or force-merge N-2
  indices.

• File-format compatibility only. File-format compatibility enables
  reading the content of old indices, but not more. Everything that is
  done on top of file formats like analysis or the encoding of length
  normalization factors is not guaranteed and only supported on a
  best-effort basis.

The reason I came up with these limitations is because I wanted to
make the scope minimal in order to retain Lucene's ability to move
forward. If there is consensus to move forward with this, I would like
to target Lucene 9.0 with this change.

This is a follow-up of the mailinglist thread here

Issue Links

• is related to: SOLR-12281 Document the new index back-compat guarantees
• links to: GitHub Pull Request #2204
• links to: GitHub Pull Request #2212
• links to: GitHub Pull Request #2221