[SLIDER-718] Slider client to support remote file operations via WebHDFS

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: Slider 0.70
• Fix Version/s: Slider 2.0.0
• Component/s: client
• Sprint: Slider December #1
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 10/Dec/14 14:56
• Updated: 29/Apr/15 16:55

Description

If slider is to work long-haul, it will need to work (client-side) with the WebHDFS HDFS API -yet potentially set up paths/configs & yarn resources to bond to the hdfs:// URLs in the destination cluster.

The issue here is not webhdfs directly (simple testing), but the possible difference between client-side URLs and in-cluster paths.

Whatever other YARN apps (will) do here, slider must copy.