[FLINK-14033] Distributed caches are not registered in Yarn Per Job Cluster Mode

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.9.0
• Fix Version/s: 1.10.0
• Component/s: Deployment / YARN
• Labels: pull-request-available
• Assignee: Zhenqiu Huang
• Reporter: Zhenqiu Huang
• Created: 09/Sep/19 16:39
• Updated: 09/Apr/20 16:20
• Resolved: 03/Dec/19 13:19

Description

CacheFiles in StreamExecutionEnvironment is not used in Job Submission in the Yarn per job cluster mode. Compare to the job submission in session cluster mode that will upload distributed cache files onto http server in application master, we should get the cache files in job graph and register them into blob store in YarnJobClusterEntrypoint.

Issue Links

• blocks: FLINK-14019 Python environment and dependency management
• causes: FLINK-15055 YarnDistributedCacheITCase does not generate logs.
• is duplicated by: FLINK-10370 DistributedCache does not work in job cluster mode.
• links to: GitHub Pull Request #9711