[HDFS-13988] Allow for Creating Temporary Files

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.2.0
• Component/s: hdfs, hdfs-client
• Assignee: Unassigned
• Reporter: David Mollitor
• Created: 12/Oct/18 16:25
• Updated: 12/Oct/18 17:53
• Resolved: 12/Oct/18 17:53

Description

Allow clients to create temporary files in HDFS.

FileSystem.java

public FSDataOutputStreamBuilder createFile(Path path);
public FSDataOutputStreamBuilder appendFile(Path path);

// New feature
public FSDataOutputStreamBuilder createTempFile(Path path, TimeUnit unit, long ttl);

This would register the file with the NameNode to have a TTL. The NameNode would be responsible for deleting the files after some custom TTL. This would be helpful in the case that a MapReduce (or Spark) job fails during execution and does not properly clean up after itself.

Issue Links

• duplicates: HDFS-6382 HDFS File/Directory TTL