[MAPREDUCE-4574] Fix TotalOrderParitioner to work with non-WritableComparable key types

• Type: Bug
• Status: Resolved
• Priority: Trivial
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 3.0.0-alpha1
• Component/s: client
• Hadoop Flags: Reviewed
• Assignee: Harsh J
• Reporter: Harsh J
• Created: 22/Aug/12 13:45
• Updated: 12/May/16 18:23
• Resolved: 09/Oct/12 09:59

Description

The current TotalOrderPartitioner class will not work with an alternative serialization library such as Avro.

To make it work, we may edit the readPartitions bits in it to support non-WritableComparable keys and also remove the WritableComparable check in the class types definition.

That is, since we do not use the values at all (NullWritable), we may as well do:

  private K[] readPartitions(FileSystem fs, Path p, Class<K> keyClass,
      Configuration conf) throws IOException {
    …
    while ((key = (K) reader.next(key)) != null) {
      parts.add(key);
      key = ReflectionUtils.newInstance(keyClass, conf);
    }
    …
  }