Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2072

Allow multiple instances of same local job to run simutaneously on the same machine

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: job submission
    • Labels:
      None

      Description

      On the same (build) machine, there may be multiple instances of same local job running - e.g. same unit test from snapshot build and release build.

      For each build project on our build machine, there is environment variable with unique value defined.

      In JobClient.submitJobInternal(), there is following code:
      JobID jobId = jobSubmitClient.getNewJobId();
      Path submitJobDir = new Path(getSystemDir(), jobId.toString());

      The above code doesn't handle the scenario described previously and often leads to the following failure:
      Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/tmp/hadoop-build/mapred/system/job_local_0002': No such file or directory
      at org.apache.hadoop.util.Shell.runCommand(Shell.java:195)
      at org.apache.hadoop.util.Shell.run(Shell.java:134)
      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:354)
      at org.apache.hadoop.util.Shell.execCommand(Shell.java:337)
      at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:492)
      at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:484)
      at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286)
      at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:308)
      at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:614)
      at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:802)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771)
      at org.apache.hadoop.mapred.HadoopClient.runJob(HadoopClient.java:177)

      One solution would be to incorporate the value of the underlying environment variable into either NewJobId or SystemDir so that there is no conflict.

        Activity

        Ted Yu created issue -
        Hide
        Allen Wittenauer added a comment -

        Stale. Probably.

        Show
        Allen Wittenauer added a comment - Stale. Probably.
        Allen Wittenauer made changes -
        Field Original Value New Value
        Resolution Fixed [ 1 ]
        Status Open [ 1 ] Resolved [ 5 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        1413d 19h 41m 1 Allen Wittenauer 30/Jul/14 23:58

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development