Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14846

Wrong shell exit code if the shell process cannot be even started

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.7.1
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: util
    • Labels:
    • Target Version/s:
    • Release Note:
      Initialize shell exit code to EXIT_CODE_NOT_AVAILABLE instead of 0 to avoid hiding failures happen before setting it to the real shell sub process exit code.
    • Flags:
      Patch, Important

      Description

      Hadoop may hide shell failures (including start container and fs operation failures), such as:

      Container exit diagnostics (note the container exit code is 0):

      Exception from container-launch. Container id: container_e5620_1503888150197_2979_01_003320
      Exit code: 0
      Exception message: 
        Cannot run program "D:\data\hadoop.latest\bin\winutils.exe" (in directory "\data\yarnnm\local\usercache\hadoop\appcache\application_1503888150197_2979\container_e5620_1503888150197_2979_01_003320"): 
        CreateProcess error=2, The system cannot find the file specified
        Stack trace: java.io.IOException: Cannot run program "D:\data\hadoop.latest\bin\winutils.exe" (in directory "\data\yarnnm\local\usercache\hadoop\appcache\application_1503888150197_2979\container_e5620_1503888150197_2979_01_003320"): CreateProcess error=2, The system cannot find the file specified
          at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
          at org.apache.hadoop.util.Shell.runCommand(Shell.java:517)
          at org.apache.hadoop.util.Shell.run(Shell.java:490)
          at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:756)
          at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212)
          at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:329)
          at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:86)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)
      

        Attachments

        1. HADOOP-14846.001.patch
          3 kB
          Yuqi Wang
        2. HADOOP-14846.002.patch
          3 kB
          Yuqi Wang

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              yqwang Yuqi Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: