Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3123

Symbolic links with special chars causing container/task.sh to fail

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None

      Description

      the following job throws an exception when you have the special characters in it.

      hadoop jar hadoop-streaming.jar -Dmapreduce.job.acl-view-job=* -Dmapreduce.job.queuename=queue1 -files file:///homes/user/hadoop/Streaming/data/streaming-980//InputDir#testlink!@$&*()-_+= -input Streaming/streaming-980/input.txt -mapper 'xargs cat' -reducer cat -output Streaming/streaming-980/Output -jobconf mapred.job.name=streamingTest-980 -jobconf mapreduce.job.acl-view-job=*

      Exception:
      2011-09-27 20:58:48,903 INFO org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: launchContainer:
      [container-executor, hadoopuser, 1, application_1317077272567_0239,
      container_1317077272567 0239_01_000001,
      tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001,
      tmp/mapred-local/nmPrivate/application_1317077272567_0239/container_1317077272567_0239_01 000001/task.sh,
      tmp/mapred-local/nmPrivate/container_1317077272567_0239_01_000001/container_1317077272567_0239_01_000001.tokens]1109221111-tests.jar:hadoop-mapreduce-p2011-09-27
      20:58:48,944 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code from container is : 2
      2011-09-27
      20:58:48,946 WARN org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exception from container-launch :

      org.apache.hadoop.util.Shell$ExitCodeException:
      /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
      line 26: syntax error near unexpected token `-_+='
      /tmp/mapred-local/usercache/hadoopuser/appcache/application_1317077272567_0239/container_1317077272567_0239_01_000001/task.sh:
      line 26: `ln -sf /tmp/mapred-local/usercache/hadoopqa/filecache/-1888139433818483070/InputDir test
      ink!@$&*()-_+='kson-jaxrs-1.7.1.jar:/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.7.3/jackson-mapper-asl-1.7.3.jar:.m2/repository/org/codehaus/jackson/jackson-xc/1.7.1/jackson-xc-1.7.1.jar:

      at org.apache.hadoop.util.Shell.runCommand(Shell.java:261)

      at org.apache.hadoop.util.Shell.run(Shell.java:188)

      at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:381)

      at org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor.launchContainer(LinuxContainerExecutor.java:174)

      at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:197)

      at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:62)
      at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
      at java.util.concurrent.FutureTask.run(FutureTask.java:138)
      at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      at java.lang.Thread.run(Thread.java:619)
      2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor:
      2011-09-27 20:58:48,951 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Processing
      container_1317077272567_0239_01_000001 of type UPDATE_DIAGNOSTICS_MSG
      2011-09-27 20:58:48,951 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch:
      Container exited with a non-zero exit code 2

      1. MR-3123.1.patch
        4 kB
        Hitesh Shah
      2. MR-3123.2.patch
        5 kB
        Hitesh Shah
      3. MR-3123.3.patch
        5 kB
        Hitesh Shah

        Activity

        Thomas Graves created issue -
        Hitesh Shah made changes -
        Field Original Value New Value
        Assignee Hitesh Shah [ hitesh ]
        Hitesh Shah made changes -
        Attachment MR-3123.1.patch [ 12498255 ]
        Hitesh Shah made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Mahadev konar made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hitesh Shah made changes -
        Attachment MR-3123.2.patch [ 12498261 ]
        Hitesh Shah made changes -
        Attachment MR-3123.3.patch [ 12498264 ]
        Hitesh Shah made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Arun C Murthy made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Hitesh Shah
            Reporter:
            Thomas Graves
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development