[HADOOP-16242] ABFS: add bufferpool to AbfsOutputStream

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0
• Component/s: fs/azure
• Target Version/s: 3.2.1
• Assignee: Da Zhou
• Reporter: Da Zhou
• Created: 09/Apr/19 23:19
• Updated: 07/Jun/19 14:12
• Resolved: 29/Apr/19 12:30

Description

In current AbfsOutPutStream, new temporary buffers always get allocated when uploading the data to server. These big chunk creates huge pressure on the GC, and in some extreme cases that GC doesn't happen in time(observed in some customer's env), it also lead to OutOfMemory issue easily.
Hence we need to replace the current implementation with a bufferpool.

Issue Links

• is related to: HADOOP-16353 Backport ABFS changes from trunk to branch-3.2
• links to: GitHub Pull Request #776