[FLINK-24888] Run jobgraph deserialization in separate thread

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Component/s: Runtime / REST
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 12/Nov/21 13:53
• Updated: 26/Nov/21 10:00
• Resolved: 26/Nov/21 10:00

Description

The JobSubmitHandler deserializers submitted job graphs. During the deserialization pretty much anything can happen, including access to thread locals and such.
Since the deserializing thread is long-lived it may thus keep a reference to the user-code around for a long time.

We should guard against this by spawning a separate thread to deserialize the jobgraph; this would prevent it since the thread can terminate after serialization.

Issue Links

• is related to: FLINK-25022 ClassLoader leak with ThreadLocals on the JM when submitting a job through the REST API