[HADOOP-11525] FileSystem should expose some performance characteristics for caller (e.g., FsShell) to choose the right algorithm.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.6.0
• Component/s: tools
• Target Version/s: 2.7.0
• Assignee: Lei (Eddy) Xu
• Reporter: Lei (Eddy) Xu
• Created: 29/Jan/15 22:05
• Updated: 10/Apr/15 20:04
• Resolved: 02/Feb/15 18:46

Description

When running hadoop fs -put, FsShell creates a .COPYING. file on the target directory, and then renames it to target file when the write is done. However, for some targeted systems, such as S3, Azure and Swift, a partial failure write request (i.e., PUT) has not side effect, while the rename operation is expensive.

FileSystem should expose some characteristics so that the operation such as CommandWithDestination#copyStreamToTarget() can detect and choose the right way to do.

Issue Links

• duplicates: HADOOP-9565 Add a Blobstore interface to add to blobstore FileSystems
• is duplicated by: HADOOP-11281 Add flag to fs.shell to skip _COPYING_ file
• is related to: HADOOP-9565 Add a Blobstore interface to add to blobstore FileSystems