[HADOOP-13853] S3ADataBlocks.DiskBlock to lazy create dest file for faster 0-byte puts

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 2.8.0
• Component/s: fs/s3
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 01/Dec/16 15:32
• Updated: 25/Jul/18 00:33
• Resolved: 25/Jul/18 00:33

Description

Looking at traces of work, there's invariably a PUT of a _SUCCESS at the end, which, with disk output, adds the overhead of creating, writing to and then reading a 0 byte file.

With a lazy create, the creation could be postponed until the first write, with special handling in the startUpload() operation to return a null stream, rather than reopen the file. Saves on some disk IO: create, read, delete

Issue Links

• is related to: HADOOP-13293 add a special 0 byte input stream for empty blobs