Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-794

Use Avro serialization in Pig

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.2.0
    • None
    • impl
    • None

    Description

      We would like to use Avro serialization in Pig to pass data between MR jobs instead of the current BinStorage. Attached is an implementation of AvroBinStorage which performs significantly better compared to BinStorage on our benchmarks.

      Attachments

        1. avro-0.1-dev-java_r765402.jar
          101 kB
          Rakesh Setty
        2. AvroStorage_2.patch
          54 kB
          Jeff Zhang
        3. AvroStorage_3.patch
          54 kB
          Jeff Zhang
        4. AvroStorage_4.patch
          37 kB
          Jeff Zhang
        5. AvroStorage.patch
          28 kB
          Rakesh Setty
        6. AvroTest.java
          2 kB
          Jeff Zhang
        7. jackson-asl-0.9.4.jar
          148 kB
          Rakesh Setty
        8. PIG-794.patch
          29 kB
          Giridharan Kesavan

        Activity

          People

            dvryaboy Dmitriy V. Ryaboy
            serakesh Rakesh Setty
            Votes:
            0 Vote for this issue
            Watchers:
            18 Start watching this issue

            Dates

              Created:
              Updated: