Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3122

-libjars with streaming does not create symlink in the cwd of task

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.20.205.0
    • Fix Version/s: None
    • Component/s: contrib/streaming
    • Labels:
      None

      Description

      When libjars option is used with streaming, the symlink to the jar file is not created in the working dir of the task. Any map/reduce tasks which uses this jar fails with NoClassDefFoundError

      For e.g:

      $ hadoop jar hadoop-streaming.jar -libjars /home/ramya/HelloWorld.jar -input input.txt -output Out -mapper "mapper.sh" -reducer NONE -file /home/ramya/mapper.sh

      $ cat /home/ramya/mapper.sh
      #!/bin/sh

      cat &> /dev/null
      java -cp HelloWorld.jar HelloWorld

      $ java -cp /home/ramya/HelloWorld.jar HelloWorld
      Hello World

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rramya Ramya Sunil
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: