Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Follow up from TEZ-752.
It would be useful to allow users to specify LocalResources as just a simple path - rather than the current requirement of a YARN local resource.
Something along the lines of
Vertex.addResource(String name, Path hdfsPath)
Vertex.addResource(String name, Path hdfsPath, AdditionalOptions)
Restricting the path to HDFS only could be an option - if we want to avoid having to manage movement of files, cleanup etc.
After this TezClient would take care of obtaining Tokens, converting this over to what is required by YARN etc.