[LUCENE-6813] OfflineSorter.sort shouldn't remove the output Path up front

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 5.4, 6.0
• Lucene Fields: New
• Assignee: Michael McCandless
• Reporter: Michael McCandless
• Created: 23/Sep/15 10:29
• Updated: 28/Aug/22 14:43
• Resolved: 24/Sep/15 21:49

Description

The new BKD tree classes, and NumericRangeTree (just a 1D BKD tree),
make heavy use of OfflineSorter to build their data structures at
indexing time when the number of indexed documents is biggish.

But when I was first building them (LUCENE-6477), I hit a thread
safety issue in OfflineSorter, and at that time I just worked around
it by creating my own private temp directory each time I need to write
a BKD tree.

This workaround is sort of messy, and it causes problems with "pending
delete" files on Windows when we try to remove that temp directory,
causing test failures like http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Windows/5149/

I think instead we should fix the root cause ... i.e. make
OfflineSorter thread safe. It looks like it's simple...

Separately I'd like to somehow fix these BKD tests to catch any leaked
file handles ... I'm not sure they are today.