Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3035

Cross platform job submission no supported

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:

      Description

      When the Tez client is located on a windows machine and the Hadoop cluster is located on Linux machines, Job submission fails for Tez (same for MapReduce) with such a exception:

      java.lang.RuntimeException: Job job_1448280010118_1679 failed! Failure info: Application application_1448280010118_1679 failed 2 times due to AM Container for appattempt_1448280010118_1679_000002 exited with  exitCode: 1
      For more detailed output, check application tracking page:http://hmaster.datameer.lan:8088/proxy/application_1448280010118_1679/Then, click on links to logs of each attempt.
      Diagnostics: Exception from container-launch.
      Container id: container_e13_1448280010118_1679_02_000001
      Exit code: 1
      Exception message: /bin/bash: line 0: fg: no job control
      
      Stack trace: ExitCodeException exitCode=1: /bin/bash: line 0: fg: no job control
      	at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
      	at org.apache.hadoop.util.Shell.run(Shell.java:455)
      	at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715)
      	at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302)
      	at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      

      For MapReduce i was able to fix the problem by setting the property mapreduce.app-submission.cross-platform=true (see https://issues.apache.org/jira/browse/MAPREDUCE-4052).

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              oae Johannes Zillmann
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: