[MAPREDUCE-1332] Ant tasks for job submission

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 0.22.0

Environment

Both platforms, Linux and Windows

• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 26/Jan/09 15:30
• Updated: 02/May/13 02:29
• Resolved: 18/Feb/12 16:24

Description

Ant tasks to make it easy to work with hadoop filesystem and submit jobs.

<submit> : uploads JAR, submits job as user, with various settings

filesystem operations: mkdir, copyin, copyout, delete
-We could maybe use Ant1.7 "resources" here, and so use hdfs as a source or dest in Ant's own tasks

1. security. Need to specify user; pick up user.name from JVM as default?
2. cluster binding: namenode/job tracker (hostname,port) or url are all that is needed?
   #job conf: how to configure the job that is submitted? support a list of <property name="name" value="something"> children
3. testing. AntUnit to generate <junitreport> compatible XML files
4. Documentation. With an example using Ivy to fetch the JARs for the tasks and hadoop client.
5. Polling: ant task to block for a job finished?

Issue Links

• depends upon: MAPREDUCE-445 Web service interface to the JobTracker
• is related to: HADOOP-2778 Hadoop job submission via ant using HMRExec
• is related to: HADOOP-1508 ant Task for FsShell operations