[FLINK-33961] Hybrid Shuffle may hang when exclusive buffers per channel is set to 0

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Component/s: Runtime / Network
• Assignee: Unassigned
• Reporter: Jiang Xin
• Created: 02/Jan/24 02:39
• Updated: 02/Jan/24 07:15
• Resolved: 02/Jan/24 07:15

Description

I found that the Hybrid Shuffle without enabling new mode may hang when exclusive-buffers-per-channel is set to 0. It can be reproduced by adding the following test into `HybridShuffleITCase.java` and running it.

@RepeatedTest(10)
void testHybridFullExchangesWithNonBuffersPerChannel() throws Exception {
    final int numRecordsToSend = 10000;
    Configuration configuration = configureHybridOptions(getConfiguration(), false);
    configuration.set(
            NettyShuffleEnvironmentOptions.NETWORK_HYBRID_SHUFFLE_ENABLE_NEW_MODE, false);
    configuration.set(NETWORK_BUFFERS_PER_CHANNEL, 0);
    JobGraph jobGraph = createJobGraph(numRecordsToSend, false, configuration);
    executeJob(jobGraph, configuration, numRecordsToSend);
}