[LUCENE-9003] Should FilterDirectoryReader compute numDocs lazily?

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 8.4
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Adrien Grand
• Created: 10/Oct/19 12:59
• Updated: 28/Aug/22 15:51
• Resolved: 14/Oct/19 18:19

Description

FilterDirectoryReader extends BaseCompositeReader, which computes both maxDoc and numDocs eagerly in its constructor by summing up these values across all sub leaves.

This is problematic for readers that hide additional documents. Computing numDocs on such leaf readers usually requires iterating over all live documents to count them. This makes creating a FilterDirectoryReader on top run in linear time, which has caused several performance bugs to us over time. This is especially frustrating given that numDocs is a rarely used index statistic.

I think computing numDocs lazily would be less surprising?

Issue Links

• links to: GitHub Pull Request #939