[LUCENE-9115] NRTCachingDirectory may put large files in the cache

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 8.5
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Adrien Grand
• Created: 06/Jan/20 13:42
• Updated: 28/Aug/22 15:55
• Resolved: 28/Jan/20 19:10

Description

NRTCachingDirectory assumes that the length of a file to write is 0 if there is no merge info or flush info. This is not correct as there are situations when Lucene might write very large files that have neither of them, for instance:

• Stored fields are written on the fly with IOContext.DEFAULT (which doesn't have flush or merge info) and without taking any of the IndexWriter buffer, so gigabytes could be written before a flush happens.
• BKD trees are merged with IOContext.DEFAULT.

Issue Links

• links to: GitHub Pull Request #1145