[FLINK-35587] Job fails with "The read buffer is null in credit-based input channel" on TPC-DS 10TB benchmark

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.0
• Component/s: Runtime / Network
• Assignee: Yuxin Tan
• Reporter: Junrui Li
• Created: 13/Jun/24 05:41
• Updated: 24/Jun/24 07:43
• Resolved: 24/Jun/24 07:42

Description

While running TPC-DS 10TB benchmark on the latest master branch locally, I've encountered a failure in certain queries, specifically query 75, resulting in the error "The read buffer is null in credit-based input channel".

Using a binary search approach, I identified the offending commit as FLINK-33668. After reverting FLINK-33668 and subsequent commits, the issue disappears. Re-applying FLINK-33668 to the branch re-introduces the error.

Please see the attached image for the error stack trace.

Issue Links

• is caused by: FLINK-33668 Decoupling Shuffle network memory and job topology
• links to: GitHub Pull Request #24968
• mentioned in: Page Loading...