[HADOOP-3400] Facilitate creation of temporary files in HDFS

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.18.0
• Hadoop Flags: Reviewed
• Assignee: Dhruba Borthakur
• Reporter: Dhruba Borthakur
• Created: 16/May/08 01:14
• Updated: 08/Jul/09 16:43
• Resolved: 22/May/08 05:03

Description

There are a set of applications that use HDFS to create temporary files. The application would ideally like these files to be automatically deleted when the application process exits. This is similar to the File.deleteOnExit() in the Java API.

One proposal is to add a new method in FileSystem

public void deleteOnExit(Path)

This API requests that the file or directory denoted by this abstract pathname be deleted when the virtual machine terminates. Deletion will be attempted only for normal termination of the virtual machine, as defined by the Java Language Specification. Once deletion has been requested, it is not possible to cancel the request. This method should therefore be used with care.

This method can be implemented entirely in the client side code, e.g. FileSystem.java will keep a cache of all the pathnames specified by the above API. FileSystem.close will invoke delete() on all the pathnames found in the cache.

Issue Links

• is related to: HADOOP-2815 Allowing processes to cleanup dfs on shutdown