Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-815

Add AvroInputFormat and AvroOutputFormat so that hadoop can use Avro Serialization

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • None
    • None
    • None
    • None

    Description

      MapReduce needs AvroInputFormat similar to other InputFormats like TextInputFormat to be able to use avro serialization in hadoop. Similarly AvroOutputFormat is needed.

      Attachments

        1. MAPREDUCE-815.patch
          74 kB
          Aaron Kimball
        2. MAPREDUCE-815.2.patch
          74 kB
          Aaron Kimball
        3. MAPREDUCE-815.3.patch
          75 kB
          Aaron Kimball
        4. MAPREDUCE-815.4.patch
          78 kB
          Aaron Kimball
        5. MAPREDUCE-815.5.patch
          88 kB
          Aaron Kimball

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. HIVE-895 Add SerDe for Avro serialized data

          • Major - Major loss of function.
          • Closed
          depends upon

          New Feature - A new feature of the product, which has yet to be developed. HADOOP-6497 Introduce wrapper around FSDataInputStream providing Avro SeekableInput interface

          • Major - Major loss of function.
          • Closed
          duplicates

          New Feature - A new feature of the product, which has yet to be developed. AVRO-493 hadoop mapreduce support for avro data

          • Major - Major loss of function.
          • Closed
          is blocked by

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1126 shuffle should use serialization to get comparator

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-6492 Make avro serialization APIs public

          • Major - Major loss of function.
          • Closed
          is related to

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-1634 add input and output formats for Avro value wrappers

          • Major - Major loss of function.
          • Resolved
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-1360 Record skipping should work with more serializations

          • Major - Major loss of function.
          • Open

          Activity

            People

              kimballa Aaron Kimball
              ravidotg Ravi Gummadi
              Votes:
              2 Vote for this issue
              Watchers:
              25 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: