[FLINK-29431] Exceptions during job graph serialization lock up client

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.16.0
• Fix Version/s: 1.16.0
• Component/s: Client / Job Submission
• Labels: pull-request-available
• Assignee: Weijie Guo
• Reporter: Chesnay Schepler
• Created: 27/Sep/22 09:44
• Updated: 28/Sep/22 09:24
• Resolved: 28/Sep/22 09:24

Description

The parallel serialization introduced in FLINK-26675 does not properly handle errors.

StreamConfig#triggerSerializationAndReturnFuture returns a future that is never completed if the serialization fails, because all we do is throw an exception but that isn't propagated anywhere.

As a result StreamingJobGraphGenerator#createJobGraph blocks indefinitely.

Issue Links

• is caused by: FLINK-26675 Parallelized heavy serialization operations in StreamingJobGraphGenerator
• links to: GitHub Pull Request #20912