Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-794

Use Avro serialization in Pig

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.2.0
    • Fix Version/s: None
    • Component/s: impl
    • Labels:
      None

      Description

      We would like to use Avro serialization in Pig to pass data between MR jobs instead of the current BinStorage. Attached is an implementation of AvroBinStorage which performs significantly better compared to BinStorage on our benchmarks.

        Attachments

        1. AvroStorage_4.patch
          37 kB
          Jeff Zhang
        2. AvroStorage_3.patch
          54 kB
          Jeff Zhang
        3. AvroTest.java
          2 kB
          Jeff Zhang
        4. AvroStorage_2.patch
          54 kB
          Jeff Zhang
        5. avro-0.1-dev-java_r765402.jar
          101 kB
          Rakesh Setty
        6. PIG-794.patch
          29 kB
          Giridharan Kesavan
        7. AvroStorage.patch
          28 kB
          Rakesh Setty
        8. jackson-asl-0.9.4.jar
          148 kB
          Rakesh Setty

          Activity

            People

            • Assignee:
              dvryaboy Dmitriy V. Ryaboy
              Reporter:
              serakesh Rakesh Setty
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated: