[HADOOP-18546] disable purging list of in progress reads in abfs stream closed

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.3.2, 3.3.3, 3.3.4
• Fix Version/s: 3.4.0, 3.3.5
• Component/s: fs/azure
• Labels: pull-request-available
• Target Version/s: 3.3.5
• Hadoop Flags: Reviewed
• Assignee: Pranav Saxena
• Reporter: Steve Loughran
• Created: 30/Nov/22 14:27
• Updated: 27/Jan/24 09:02
• Resolved: 15/Dec/22 17:14

Description

turn off the prune of in progress reads in ReadBufferManager::purgeBuffersForStream

this will ensure active prefetches for a closed stream complete. they wiill then get to the completed list and hang around until evicted by timeout, but at least prefetching will be safe.

When backporting

• include the followup test fix patch
• include HADOOP-18577.

Issue Links

• is depended upon by: HADOOP-18577 ABFS: add probes of readahead fix
• relates to: HADOOP-18589 Create a new config for ABFS read-ahead
• links to: GitHub Pull Request #5176
• links to: GitHub Pull Request #5198
• links to: GitHub Pull Request #5205
• links to: GitHub Pull Request #5528
• links to: GitHub Pull Request #5529
• links to: GitHub Pull Request #5530
• links to: GitHub Pull Request #5531