Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1126

shuffle should use serialization to get comparator

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Won't Fix
    • None
    • None
    • task
    • None
    • Reviewed

    Description

      Currently the key comparator is defined as a Java class. Instead we should use the Serialization API to create key comparators. This would permit, e.g., Avro-based comparators to be used, permitting efficient sorting of complex data types without having to write a RawComparator in Java.

      Attachments

        1. m-1126-2.patch
          2 kB
          Owen O'Malley
        2. m-1126-3.patch
          2 kB
          Owen O'Malley
        3. MAPREDUCE-1126.2.patch
          107 kB
          Aaron Kimball
        4. MAPREDUCE-1126.3.patch
          107 kB
          Aaron Kimball
        5. MAPREDUCE-1126.4.patch
          108 kB
          Aaron Kimball
        6. MAPREDUCE-1126.5.patch
          112 kB
          Aaron Kimball
        7. MAPREDUCE-1126.6.patch
          120 kB
          Aaron Kimball
        8. MAPREDUCE-1126.patch
          75 kB
          Thomas White
        9. MAPREDUCE-1126.patch
          90 kB
          Aaron Kimball

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kimballa Aaron Kimball
            cutting Doug Cutting
            Votes:
            0 Vote for this issue
            Watchers:
            38 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment