Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-617

Streaming should not throw java.lang.RuntimeException and ERROR while displaying help


    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.20.2
    • Component/s: contrib/streaming
    • Labels:


      Run streaming command as -:
      bin/hadoop jar contrib/streaming/hadoop-*-streaming.jar -help
      it will dislay following -:
      ERROR streaming.StreamJob: Missing required option -input
      Usage: $HADOOP_HOME/bin/hadoop [--config dir] jar \
      $HADOOP_HOME/hadoop-streaming.jar [options]
      -input <path> DFS input file(s) for the Map step
      -output <path> DFS output directory for the Reduce step
      -mapper <cmd|JavaClassName> The streaming command to run
      -combiner <JavaClassName> Combiner has to be a Java class
      -reducer <cmd|JavaClassName> The streaming command to run
      -file <file> File/dir to be shipped in the Job jar file
      -dfs <h>|local Optional. Override DFS configuration
      -jt <h>|local Optional. Override JobTracker configuration
      -additionalconfspec specfile Optional.
      -inputformat TextInputFormat(default)|SequenceFileAsTextInputFormat|JavaClassName Optional.
      -outputformat TextOutputFormat(default)|JavaClassName Optional.
      -partitioner JavaClassName Optional.
      -numReduceTasks <num> Optional.
      -inputreader <spec> Optional.
      -jobconf <n>=<v> Optional. Add or override a JobConf property
      -cmdenv <n>=<v> Optional. Pass env.var to streaming commands
      -mapdebug <path> Optional. To run this script when a map task fails
      -reducedebug <path> Optional. To run this script when a reduce task fails
      -cacheFile fileNameURI
      -cacheArchive fileNameURI

      For more details about these options:
      Use $HADOOP_HOME/bin/hadoop jar build/hadoop-streaming.jar -info

      Exception in thread "main" java.lang.RuntimeException:
      at org.apache.hadoop.streaming.StreamJob.fail(StreamJob.java:542)
      at org.apache.hadoop.streaming.StreamJob.exitUsage(StreamJob.java:481)
      at org.apache.hadoop.streaming.StreamJob.parseArgv(StreamJob.java:203)
      at org.apache.hadoop.streaming.StreamJob.go(StreamJob.java:105)
      at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:33)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:585)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:155)

      It should not display execption and -input and error about missin -input option


        Karam Singh created issue -
        Owen O'Malley made changes -
        Field Original Value New Value
        Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
        Key HADOOP-2877 MAPREDUCE-617
        Issue Type Improvement [ 4 ] Bug [ 1 ]
        Component/s contrib/streaming [ 12312905 ]
        Component/s contrib/streaming [ 12310972 ]
        Karam Singh made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Fix Version/s 0.20.2 [ 12314205 ]
        Resolution Fixed [ 1 ]


          • Assignee:
            Karam Singh
          • Votes:
            0 Vote for this issue
            0 Start watching this issue


            • Created: