[LUCENE-8324] Unreferenced files of dropped segments should be released

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 7.4, 8.0
• Component/s: core/index
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Nhat Nguyen
• Created: 20/May/18 01:16
• Updated: 28/Aug/22 15:30
• Resolved: 21/May/18 18:38

Description

This has the side-effect that flushed segments that are 100% hard deleted are also
cleaned up right after they are flushed, previously these segments were sticking
around for a while until they got picked for a merge or received another delete.

Since LUCENE-8253, a fully deleted segment is dropped immediately when it's flushed, however, its files might be kept around even after a commit. In other words, we may have unreferenced files which are retained by Deleter.

I am not entirely sure if we should fix this but it's nice to have a consistent content between current files and commit points as before.

I attached a failed test for this.

/cc simonw