[SPARK-26674] Consolidate CompositeByteBuf when reading large frame

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core
• Assignee: liupengcheng
• Reporter: liupengcheng
• Created: 21/Jan/19 11:33
• Updated: 26/Feb/19 00:44
• Resolved: 26/Feb/19 00:41

Description

Currently, TransportFrameDecoder will not consolidate the buffers read from network which may cause memory waste. Actually, bytebuf's writtenIndex is far less than it's capacity  in most cases, so we can optimize it by doing consolidation.

 

Issue Links

• links to: GitHub Pull Request #23602