[LUCENE-9334] Require consistency between data-structures on a per-field basis

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 9.0
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Adrien Grand
• Created: 21/Apr/20 09:08
• Updated: 28/Aug/22 16:02
• Resolved: 16/Jul/21 14:54

Description

Follow-up of https://lists.apache.org/thread.html/r747de568afd7502008c45783b74cc3aeb31dab8aa60fcafaf65d5431%40%3Cdev.lucene.apache.org%3E.

We would like to start requiring consitency across data-structures on a per-field basis in order to make it easier to do the right thing by default: range queries can run faster if doc values are enabled, sorted queries can run faster if points by indexed, etc.

This would be a big change, so it should be rolled out in a major.

Strict validation is tricky to implement, but we should still implement best-effort validation:

• Documents all use the same data-structures, e.g. it is illegal for a document to only enable points and another document to only enable doc values,
• When possible, check whether values are consistent too.

Issue Links

• causes: SOLR-15356 Lucene requires consistency across fields
• contains: LUCENE-9078 Term vectors options should not be configurable per-doc
• links to: GitHub Pull Request #11
• links to: GitHub Pull Request #2186