[REEF-456] Make job submission folder prefix configurable

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 0.10, 0.11
• Fix Version/s: 0.12
• Component/s: REEF Runtime YARN
• Assignee: Beysim Sezgin
• Reporter: Beysim Sezgin
• Created: 06/Jul/15 23:21
• Updated: 09/Jul/15 19:22
• Resolved: 09/Jul/15 19:22

Description

Our users want to specify job submission folder on DFS due to permission and other issues. We currently allow this with JobSubmissionDirectory named Parameter. But this causes users to deal with uniqueness issues. We should make folder prefix configurable and append job id (IE. yarn application Id) to form full path. This also makes it easier to correlate folder location with yarn app id for debugging purposes.