[FLINK-12343] Allow set file.replication in Yarn Configuration

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Command Line Client, Deployment / YARN
• Labels: pull-request-available
• Assignee: Zhenqiu Huang
• Reporter: Zhenqiu Huang
• Created: 26/Apr/19 17:49
• Updated: 19/Jun/20 23:02
• Resolved: 20/Feb/20 16:35

Description

Currently, FlinkYarnSessionCli upload jars into hdfs with default 3 replications. From our production experience, we find that 3 replications will block big job (256 containers) to launch, when the HDFS is slow due to big workload for batch pipelines. Thus, we want to make the factor customizable from FlinkYarnSessionCli by adding an option.

Issue Links

• links to: GitHub Pull Request #8303
• links to: GitHub Pull Request #10980