[SOLR-8273] deprecate implicitly uninverted fields, force people to either use docValues, or be explicit that they want query time uninversion

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Schema and Analysis
• Assignee: Unassigned
• Reporter: Chris M. Hostetter
• Created: 10/Nov/15 20:14
• Updated: 06/Nov/18 04:17

Description

once upon a time, there was nothing we could do to stop people from using the FieldCache - even if they didn't realize they were using it.

Then DocValues was added - and now people have a choice: they can set docValues=true on a field/fieldtype and know that when they do functions/sorting/faceting on that field, it won't require a big hunk of ram and a big stall everytime a reader was reopened. But it's easy to overlook when clients might be doing something that required the FieldCache w/o realizing it – and there is no way to stop them, because Solr automatically uses UninvertingReader under the covers and automatically allows every field to be uninverted in this way.

we should change that.

---

Straw man proposal...

• introduce a new boolean fieldType/field property uninvertable
• all existing FieldType classes should default to uninvertable==false
• a field or fieldType that contains indexed="false" uninvertable="true" should be an error.
• the Schema version value should be incremented, such that any Schema with an older version is treated as if every field with docValues==false has an implict uninvertable="true" on it.
• the Map passed to UninvertedReader should now only list items that have an effective value of uninvertable==true
• sample schemas should be updated to use docValues on any field where the examples using those schemas suggest using those fields in that way (ie: sorting, faceting, etc...)

Issue Links

• is related to: SOLR-8272 Allow user to specify actions allowed for each field

Sub-Tasks

• add an 'uninvertible' field(type) option (that defaults to "true" for backcompat): Closed
• change default for 'uninvertible' to 'false' (dependent on new schema 'version'): Open