[HADOOP-18944] s3a openfile is using readahead rather than async drain threshold to calculate drain threshold

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.3.6
• Component/s: fs/s3
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 18/Oct/23 09:59
• Updated: 25/Oct/23 18:12

Description

we are using the wrong default value for the async drain threshold in org.apache.hadoop.fs.s3a.impl.OpenFileSupport

       builderSupport.getPositiveLong(ASYNC_DRAIN_THRESHOLD,
                defaultReadAhead))

this means the drain threshold isn't being taken up in random io unless you have a very small readahead

Issue Links

• relates to: HADOOP-16202 Enhance openFile() for better read performance against object stores
• links to: GitHub Pull Request #6203