Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-525

Need raw comparators for hadoop record types

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.6.0
    • 0.10.0
    • record
    • None

    Description

      Raw comparators are not generated for types that are generated with the Hadoop record framework. This could have a substantial performance impact when using hadoop record generated types in Map/Reduce. The record i/o framework should auto-generate raw comparators for types.

      Comparison for hadoop record i/o types is defined to be member wise comparison of objects. A possible implementation could only deserialize one member from each object at a time, compare them and either return or move on to the next member if the values are equal.

      Attachments

        1. raw-comparators.patch
          58 kB
          Milind Barve
        2. TypeBuilder.java
          32 kB
          Trevor Strohman
        3. TypeBuilder-support.tar
          30 kB
          Trevor Strohman
        4. WordCountType.java
          18 kB
          Trevor Strohman

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            milindb Milind Barve
            sameerp Sameer Paranjpye
            Votes:
            3 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment