Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1126

shuffle should use serialization to get comparator

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: task
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently the key comparator is defined as a Java class. Instead we should use the Serialization API to create key comparators. This would permit, e.g., Avro-based comparators to be used, permitting efficient sorting of complex data types without having to write a RawComparator in Java.

      1. m-1126-2.patch
        2 kB
        Owen O'Malley
      2. m-1126-3.patch
        2 kB
        Owen O'Malley
      3. MAPREDUCE-1126.2.patch
        107 kB
        Aaron Kimball
      4. MAPREDUCE-1126.3.patch
        107 kB
        Aaron Kimball
      5. MAPREDUCE-1126.4.patch
        108 kB
        Aaron Kimball
      6. MAPREDUCE-1126.5.patch
        112 kB
        Aaron Kimball
      7. MAPREDUCE-1126.6.patch
        120 kB
        Aaron Kimball
      8. MAPREDUCE-1126.patch
        75 kB
        Tom White
      9. MAPREDUCE-1126.patch
        90 kB
        Aaron Kimball

        Issue Links

          Activity

            People

            • Assignee:
              Aaron Kimball
              Reporter:
              Doug Cutting
            • Votes:
              0 Vote for this issue
              Watchers:
              39 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development