[SOLR-1953] It may be possible for temporary files to accumulator until the Solr process is shut down

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.1, 1.5
• Fix Version/s: 6.4, 7.0
• Component/s: update
• Labels: file_leak memory_leak
• Assignee: Mark Miller
• Reporter: Karl Wright
• Created: 15/Jun/10 10:59
• Updated: 13/Apr/17 10:50
• Resolved: 19/Dec/16 18:28

Description

While researching SOLR-1951, the behavior of commons-fileupload in handling multipart form posts came into question. commons-fileupload creates a temporary file for the main content area of such posts, and purportedly has a background thread which cleans up these files. However, Mark Miller discovered that the javadoc in this matter may be incorrect, and that commons-fileupload may in fact just be adding files to the JVM's list of files needing cleanup on exit.

If so, this will show up in two ways: first, temporary files will accumulate in the java.io.tmpdir area. Second, non-heap memory for the JVM will slowly increase over time (since the file pointers the JVM tracks in this way are not kept in the java heap).

I will attach a potential fix; however, this ticket should be viewed as a workitem for the need for further research in this area.

Issue Links

• is related to: FILEUPLOAD-189 DiskFileItemFactory use of FileCleaningTracker is documented or coded wrong
• relates to: SOLR-1951 extractingUpdateHandler doesn't close socket handles promptly, and indexing load tests eventually run out of resources