[FLINK-24469] Incorrect calcualtion of the buffer size in case of channel data skew

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.14.0
• Fix Version/s: 1.14.3, 1.15.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Anton Kalashnikov
• Reporter: Anton Kalashnikov
• Created: 07/Oct/21 09:30
• Updated: 15/Dec/21 01:44
• Resolved: 18/Oct/21 12:10

Description

If only a couple out of 100 input channels are active then there is a huge throughput drop is observed. I suppose it happens because of the line inside of SingleInputGate#getBuffersInUseCount:

Math.max(1, channel.getBuffersInUseCount());

Right now, I don't think that it makes sense to have min value as 1, perhaps zero buffer in use is also ok.

Issue Links

• links to: GitHub Pull Request #17479