Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-617

Streaming should not throw java.lang.RuntimeException and ERROR while displaying help

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.20.2
    • contrib/streaming
    • None

    Description

      Run streaming command as -:
      bin/hadoop jar contrib/streaming/hadoop-*-streaming.jar -help
      it will dislay following -:
      ERROR streaming.StreamJob: Missing required option -input
      Usage: $HADOOP_HOME/bin/hadoop [--config dir] jar \
      $HADOOP_HOME/hadoop-streaming.jar [options]
      Options:
      -input <path> DFS input file(s) for the Map step
      -output <path> DFS output directory for the Reduce step
      -mapper <cmd|JavaClassName> The streaming command to run
      -combiner <JavaClassName> Combiner has to be a Java class
      -reducer <cmd|JavaClassName> The streaming command to run
      -file <file> File/dir to be shipped in the Job jar file
      -dfs <h>|local Optional. Override DFS configuration
      -jt <h>|local Optional. Override JobTracker configuration
      -additionalconfspec specfile Optional.
      -inputformat TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName Optional.
      -outputformat TextOutputFormat(default)|JavaClassName Optional.
      -partitioner JavaClassName Optional.
      -numReduceTasks <num> Optional.
      -inputreader <spec> Optional.
      -jobconf <n>=<v> Optional. Add or override a JobConf property
      -cmdenv <n>=<v> Optional. Pass env.var to streaming commands
      -mapdebug <path> Optional. To run this script when a map task fails
      -reducedebug <path> Optional. To run this script when a reduce task fails
      -cacheFile fileNameURI
      -cacheArchive fileNameURI
      -verbose

      For more details about these options:
      Use $HADOOP_HOME/bin/hadoop jar build/hadoop-streaming.jar -info

      Exception in thread "main" java.lang.RuntimeException:
      at org.apache.hadoop.streaming.StreamJob.fail(StreamJob.java:542)
      at org.apache.hadoop.streaming.StreamJob.exitUsage(StreamJob.java:481)
      at org.apache.hadoop.streaming.StreamJob.parseArgv(StreamJob.java:203)
      at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:105)
      at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:33)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:155)

      It should not display execption and -input and error about missin -input option

      Attachments

        Activity

          People

            Unassigned Unassigned
            karams Karam Singh
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: