[LUCENE-8031] DOCS_ONLY fields set incorrect length norms

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 8.0
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Robert Muir
• Created: 01/Nov/17 23:55
• Updated: 28/Aug/22 15:21
• Resolved: 24/Feb/18 16:22

Description

Term frequencies are discarded in the DOCS_ONLY case from the postings list but they still count against the length normalization, which looks like it may screw stuff up.

I ran some quick experiments on LUCENE-8025, by encoding fieldInvertState.getUniqueTermCount() and it seemed worth fixing (e.g. 20% or 30% improvement potentially). Happy to do testing for real, if we want to fix.

But this seems tricky, today you can downgrade to DOCS_ONLY on the fly, and its hard for me to think about that case (i think its generally screwed up besides this, but still).