Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5782

Sort Use TableOrderPartitioner cannot find the path of _partition.lst int the container cwd.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.23.10, 2.4.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Fix the sort example when TotalOrderPartitioner is specified.

      Description

      hadoop jar hadoop-mapreduce-examples.jar sort sort \
      -inFormat org.apache.hadoop.mapreduce.lib.input.KeyValueTextInputFormat \
      -outFormat org.apache.hadoop.mapreduce.lib.output.TextOutputFormat \
      -outKey org.apache.hadoop.io.Text \
      -outValue org.apache.hadoop.io.Text \
      -totalOrder 0.1 10000 10 \
      input output
      It prints cannot find the file _partition.lst in the container's cwd.
      Error: java.lang.IllegalArgumentException: Can't read partitions file
      at
      org.apache.hadoop.mapreduce.lib.partition.TotalOrderPartitioner.setConf(TotalOrderPartitioner.java:116)
      at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:70)
      at
      org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:130)
      at
      org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:642)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:710)
      at
      org.apache.hadoop.mapred.MapTask.run(MapTask.java:333)
      at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158)

        Activity

        Chao Zhang made changes -
        Assignee Chao Zhang [ combozhc ]
        Chao Zhang made changes -
        Assignee Chao Zhang [ combozhc ]
        Chao Zhang made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Chao Zhang made changes -
        Assignee Chao Zhang [ combozhc ]
        Chao Zhang made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Fix the sort example when TotalOrderPartitioner is specified.
        Affects Version/s 2.4.0 [ 12326141 ]
        Chao Zhang made changes -
        Assignee Chao Zhang [ combozhc ]
        Chao Zhang made changes -
        Field Original Value New Value
        Attachment MAPREDUCE-5782.001.patch [ 12706074 ]
        Bing Jiang created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Bing Jiang
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:

              Development