Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-815

Add AvroInputFormat and AvroOutputFormat so that hadoop can use Avro Serialization

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      MapReduce needs AvroInputFormat similar to other InputFormats like TextInputFormat to be able to use avro serialization in hadoop. Similarly AvroOutputFormat is needed.

      Attachments

        1. MAPREDUCE-815.2.patch
          74 kB
          Aaron Kimball
        2. MAPREDUCE-815.3.patch
          75 kB
          Aaron Kimball
        3. MAPREDUCE-815.4.patch
          78 kB
          Aaron Kimball
        4. MAPREDUCE-815.5.patch
          88 kB
          Aaron Kimball
        5. MAPREDUCE-815.patch
          74 kB
          Aaron Kimball

        Issue Links

        blocks

        New Feature - A new feature of the product, which has yet to be developed. HIVE-895 Add SerDe for Avro serialized data

        • Major - Major loss of function.
        • Closed
        depends upon

        New Feature - A new feature of the product, which has yet to be developed. HADOOP-6497 Introduce wrapper around FSDataInputStream providing Avro SeekableInput interface

        • Major - Major loss of function.
        • Closed
        duplicates

        New Feature - A new feature of the product, which has yet to be developed. AVRO-493 hadoop mapreduce support for avro data

        • Major - Major loss of function.
        • Closed
        is blocked by

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1126 shuffle should use serialization to get comparator

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-6492 Make avro serialization APIs public

        • Major - Major loss of function.
        • Closed
        is related to

        New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-1634 add input and output formats for Avro value wrappers

        • Major - Major loss of function.
        • Resolved
        relates to

        Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1360 Record skipping should work with more serializations

        • Major - Major loss of function.
        • Open

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            kimballa Aaron Kimball Assign to me
            ravidotg Ravi Gummadi
            Votes:
            2 Vote for this issue
            Watchers:
            25 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment