Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4793

Problem with adding resources when using both -files and -file to hadoop streaming

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.3
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      It seems when
      both -files and -file are present, it will trigger this IAE, and the error
      message is just misleading.

      hadoop jar $HADOOP_PREFIX/share/hadoop/tools/lib/hadoop-streaming.jar
      -files
      hdfs://host:port/user/foo/access.log#test
      -input 'input' -output 'output' -mapper "egrep '.*'" -file tmp.file

      Below is the error message
      12/11/02 07:37:54 INFO mapreduce.JobSubmitter: Cleaning up the staging area
      /user/haiyang/.staging/job_1351804437209_0575
      Exception in thread "main" java.lang.IllegalArgumentException: Resource name
      must be relative
      at
      org.apache.hadoop.mapreduce.v2.util.MRApps.parseDistributedCacheArtifacts(MRApps.java:383)
      at
      org.apache.hadoop.mapreduce.v2.util.MRApps.setupDistributedCache(MRApps.java:324)
      at
      org.apache.hadoop.mapred.YARNRunner.createApplicationSubmissionContext(YARNRunner.java:419)
      at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:288)

        Attachments

        1. MAPREDUCE-4793.patch
          0.9 kB
          Jason Darrell Lowe

          Activity

            People

            • Assignee:
              jlowe Jason Darrell Lowe
              Reporter:
              tgraves Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: