Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5990

If output directory can not be created, error message on stdout does not provide any clue.

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • examples

    Description

      In the following wordcount example output directory path can not be created because /temp does not exists and user has not privileges to create output path at "/".

      hadoop --config ./clustdir/ jar /homes/gogate/wordcount.jar com.XXXX.wordcount.WordCount /in-path/gogate/myfile /temp/mywc-gogate
      09/04/28 23:00:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
      09/04/28 23:00:32 INFO mapred.FileInputFormat: Total input paths to process : 1
      09/04/28 23:00:32 INFO mapred.FileInputFormat: Total input paths to process : 1
      09/04/28 23:00:33 INFO mapred.JobClient: Running job: job_200904282249_0004
      java.io.IOException: Job failed!
      at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1113)
      at com.XXXX.wordcount.WordCount.main(WordCount.java:55)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
      at org.apache.hadoop.mapred.JobShell.run(JobShell.java:54)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
      at org.apache.hadoop.mapred.JobShell.main(JobShell.java:68)

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            vgogate Suhas

            Dates

              Created:
              Updated:

              Slack

                Issue deployment