[SPARK-38683] It is unnecessary to release the ShuffleManagedBufferIterator or ShuffleChunkManagedBufferIterator or ManagedBufferIterator buffers when the client channel's connection is terminated

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.1.1, 3.1.2, 3.2.0, 3.2.1
• Fix Version/s: 3.4.0
• Component/s: Shuffle
• Assignee: weixiuli
• Reporter: weixiuli
• Created: 29/Mar/22 09:21
• Updated: 04/Apr/22 12:24
• Resolved: 01/Apr/22 02:08

Description

 It is unnecessary to release the ShuffleManagedBufferIterator or ShuffleChunkManagedBufferIterator or ManagedBufferIterator buffers when the client channel's connection is terminated, to reduce I/O operations and improve performance for the External Shuffle Service.

Issue Links

• links to: [Github] Pull Request #36000 (weixiuli)
• links to: [Github] Pull Request #36060 (weixiuli)
• links to: [Github] Pull Request #36060 (weixiuli)
• links to: [Github] Pull Request #36061 (weixiuli)