[PIG-1519] Stop relying on finalize() to delete files, close filehandles in bag implementations

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.8.0
• Assignee: Unassigned
• Reporter: Thejas Nair
• Created: 26/Jul/10 23:45
• Updated: 27/Jul/10 00:10

Description

In DefaultAbstractBag and its subclasses, the files used for spilling to disk are deleted using finalize() .
The iterators associated with these bags use DataInputStreams but don't call close on them, and the underlying FileInputStream.close() is called only through FileInputStream.finalize().

The use of finalize has performance implications and also makes it hard to predict when the resources will get freed.

WeakReferences can be used to avoid the use of finalize(). See http://java.sun.com/developer/technicalArticles/javase/finalization/ (look for "An Alternative to Finalization") .

I have marked the priority has minor because the allocation of these resources objects that have finalize happens only for large bags that spill to disk (see related jira - PIG-1516), so the performance impact of the use of finalize is not likely to be significant. Also, I haven't come across any case where we have run out of these resources because finalize() thread has not freed them yet.