[LUCENE-5329] Make DocumentDictionary and co more lenient to dirty documents

• Type: Improvement
• Status: Reopened
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.7, 6.0
• Component/s: core/search
• Lucene Fields: New, Patch Available
• Assignee: Unassigned
• Reporter: Areek Zillur
• Created: 05/Nov/13 09:00
• Updated: 15/Sep/24 22:23
• Resolved: 22/Nov/13 15:28

Description

Currently DocumentDictionary errors out whenever any document does not have value for any relevant stored fields. It would be nice to make it lenient and instead ignore the invalid documents.

Another "issue" with the DocumentDictionary is that it only allows string fields as suggestions and binary fields as payloads. When exposing these dictionaries to solr (via https://issues.apache.org/jira/browse/SOLR-5378), it is inconvenient for the user to ensure that a suggestion field is a string field and a payload field is a binary field. It would be nice to have the dictionary "just work" whenever a string/binary field is passed to suggestion/payload field. The patch provides one solution to this problem (by accepting string or binary values), though it would be great if there are any other solution to this, without making the DocumentDictionary "too flexible"