Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1068

In hadoop-0.20.0 streaming job do not throw proper verbose error message if file is not present

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.20.2
    • Component/s: contrib/streaming
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fix streaming job to show proper message if file is is not present, for -file option.

      Description

      With hadoop-0.20.0 proper error message is not thrown , when streaming job is submitted and if file is not present to be distributed with "-file" option. But with hadoop-0.18.* proper verbose message is thrown if file is not present and it is easy for the users to debug.

      For example:

      With hadoop-0.20.0:
      $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -Dmapred.job.queue.name=general -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
      Streaming Job Failed!

      With hadoop-0.18.*:
      $ hadoop jar $HADOOP_HOME/hadoop-streaming.jar -input /user/simon/test1.txt -output /user/simon/test.out -mapper "/bin/cat" -file dummy -reducer "/bin/cat"
      09/10/06 10:06:49 ERROR streaming.StreamJob: Unexpected value "Argument : /home/simon/dummy doesn't exist." found while processing -file

      1. patch-1068-1.txt
        1 kB
        Amareshwari Sriramadasu
      2. patch-1068.txt
        0.9 kB
        Amareshwari Sriramadasu

        Activity

          People

          • Assignee:
            Amareshwari Sriramadasu
            Reporter:
            Peeyush Bishnoi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development