[HADOOP-13531] S3A output streams to share a single LocalDirAllocator for round-robin drive use

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 2.8.0
• Component/s: fs/s3
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 22/Aug/16 08:54
• Updated: 10/Jan/17 18:36
• Resolved: 13/Oct/16 15:00

Description

S3AOutputStream uses LocalDirAllocator to choose a directory from the comma-separated list of buffers —but it creates a new instance for every output stream. This misses a key point of the allocator: for it to do round-robin allocation, it needs to remember the last disk written to. If a new instance is used for every file: no history.

Issue Links

• is superceded by: HADOOP-13560 S3ABlockOutputStream to support huge (many GB) file writes