[FLINK-23928] Test the buffer debloating

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.14.0
• Labels: release-testing stale-assigned
• Assignee: Chesnay Schepler
• Reporter: Anton Kalashnikov
• Created: 23/Aug/21 18:49
• Updated: 29/Sep/21 07:45
• Resolved: 29/Sep/21 07:45

Description

1. Configure the buffer debloat by setting `taskmanager.network.memory.buffer-debloat.enabled` to true
2. Start the job with a long checkpoint time
3. Check the alignment time for each subtask it should be near taskmanager.network.memory.buffer-debloat.target
4. Check the checkpoint time it should be near taskCount * taskmanager.network.memory.buffer-debloat.target
5. Try to change taskmanager.network.memory.buffer-debloat.target. Checkponint time should be changed correspondetly.
6. Check metrics estimatedTimeToConsumerBuffersMs and debloatedBufferSize which also should depend on the current debloat configuration.

 

Other configuration which can be changed for test.

• taskmanager.memory.min-segment-size(if this would be equal to taskmanager.memory.segment-size then the behaviour should be same as with disabled buffer debloat)
• taskmanager.network.memory.buffer-debloat.period
• taskmanager.network.memory.buffer-debloat.samples
• taskmanager.network.memory.buffer-debloat.threshold-percentages

Issue Links

• is blocked by: FLINK-23458 Document required number of buffers in the network stack