[HADOOP-5438] Merge FileSystem.create and FileSystem.append

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.21.0
• Component/s: fs
• Hadoop Flags: Incompatible change
• Assignee: He Yongqiang
• Reporter: He Yongqiang
• Created: 09/Mar/09 05:56
• Updated: 12/Aug/20 23:02
• Resolved: 25/May/09 05:25

Description

Currently, when a user wants to modify a file, the user first calls exists() to know if this file is already there. And then uses create() or append() according to whether the file exists or not.
the code looks like:

FSDataOutputStream out_1 = null;
if (fs.exists(path_1))
   out_1 = fs.append(path_1);
else
   out_1 = fs.create(path_1);

. On the performace side,It involes two RPCs. On the easy-of-use side, it is not very convient in contrast to the traditional open interface.

It will more complicate if there is a overwrite parameter specified. I donot know whether there is a bug about 'overwrite' in 0.19, some times it takes a long time for overwrite creates to reture. So i make the write file code with overwrite param works like:

boolean exists = fs.exists(name);
if (overwrite) {
    if (exists)
       fs.delete(name, true);
     this.out = fs.create(name, overwrite, bufferSize, replication,
				    blockSize, progress);
     this.currentRowID = 0;
 } else {
   if (!exists)
	this.out = fs.create(name, overwrite, bufferSize,
					replication, blockSize, progress);
   else
	this.out = fs.append(name, bufferSize, progress);

Some code statements there are really redundant and not needed, especialy with the delete(). But without deleting first, the overwrite takes a long time to reture.

BTW, i will create another issue about the overwrite problem. If it is not a bug at all or a duplicate, someone please close it.

Issue Links

• is blocked by: HADOOP-5596 Make ObjectWritable support EnumSet
• is related to: HADOOP-5439 FileSystem.create() with overwrite param specified sometimes takes a long time to return.
• is related to: HDFS-609 Create a file with the append flag does not work in HDFS
• is related to: HDFS-499 Fix deprecation warnings introduced by HADOOP-5438
• is related to: HADOOP-6138 eliminate the depracate warnings introduced by H-5438
• relates to: HDFS-440 javadoc warnings: broken links
• relates to: HADOOP-3252 Reduce the number of public FileSystem.create() methods