[FLINK-13993] Using FlinkUserCodeClassLoaders to load the user class in the perjob mode

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0, 1.10.0
• Fix Version/s: 1.10.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Guowei Ma
• Reporter: Guowei Ma
• Created: 06/Sep/19 15:15
• Updated: 25/Nov/21 11:56
• Resolved: 09/Dec/19 09:59

Description

Currently, Flink has the FlinkUserCodeClassLoader, which is using to load user’s class. However, the user class and the system class are all loaded by the system classloader in the perjob mode. This introduces some conflicts.

This document[1] gives a proposal that makes the FlinkUserClassLoader load the user class in perjob mode. (disscuss with Till[2])

 

[1]https://docs.google.com/document/d/1fH2Cwrrmps5RxxvVuUdeprruvDNabEaIHPyYps28WM8/edit#heading=h.815t5dodlxh7

[2] https://docs.google.com/document/d/1SUhFt1BmsGMLUYVa72SWLbNrrWzunvcjAlEm8iusvq0/edit

Issue Links

• causes: FLINK-25053 Document how to use the usrlib to load code in the user code class loader
• duplicates: FLINK-13840 Let StandaloneJobClusterEntrypoint use user code class loader
• links to: GitHub Pull Request #9715
• links to: GitHub Pull Request #9715
• links to: GitHub Pull Request #9950

Sub-Tasks

• Introduce the 'AbstractUserClassPathJobGraphRetriever' which could construct the 'user code class path' from the "job" dir.: Resolved
• Let StandaloneJobClusterEntrypoint use user code class loader: Resolved
• Let YarnJobClusterEntrypoint use user code class loader: Closed
• Let MesosJobClusterEntrypoint use user code class loader: Closed