[FLINK-20811] Support HTTP paths for yarn ship files/archives

• Type: New Feature
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: Deployment / YARN
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned
• Assignee: Unassigned
• Reporter: Xintong Song
• Created: 30/Dec/20 03:32
• Updated: 12/Dec/21 10:38

Description

Flink's Yarn integration supports shipping workload-specific local files/directories/archives to the Yarn cluster.

As discussed in FLINK-20505, it would be helpful to support directly downloading contents from HTTP paths to the Yarn cluster, so that users won't need to first download the contents locally and then upload it to the Yarn cluster.

 

Issue Links

• is related to: FLINK-20681 Support specifying the hdfs path when ship archives or files
• relates to: FLINK-20505 Yarn provided lib does not work with http paths.
• relates to: FLINK-20867 Support ship files from local/remote path to k8s pod
• mentioned in: Page Loading...