Pig
  1. Pig
  2. PIG-2854

AvroStorage doesn't work with Avro 1.7.1

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11
    • Component/s: piggybank
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      PigAvroDatumWriter fails to compile against Avro 1.7.1 with the following error:

      /home/cheolsoo/workspace/pig-trunk/contrib/piggybank/java/src/main/java/org/apache/pig/piggybank/storage/avro/PigAvroDatumWriter.java:119: resolveUnion(org.apache.avro.Schema,java.lang.Object) in org.apache.pig.piggybank.storage.avro.PigAvroDatumWriter cannot override resolveUnion(org.apache.avro.Schema,java.lang.Object) in org.apache.avro.generic.GenericDatumWriter; overridden method does not throw java.io.IOException
      protected int resolveUnion(Schema union, Object datum) throws IOException {
      

      The problem is that a new method resolveUnion() is added to GenericDatumWriter in Avro 1.6.x and 1.7.x, but this method conflicts with the one that already exists in PigAvroDatatumWriter.

      1. PIG-2854.patch
        2 kB
        Cheolsoo Park

        Activity

        Cheolsoo Park created issue -
        Hide
        Cheolsoo Park added a comment -

        Attached is a patch that changes the name of the method in PigAvroDatumWriter from resolveUnion() to resolveUnionSchema() to avoid the conflict.

        I tested against Avro 1.5.3, 1.6.3, and 1.7.1.

        Thanks!

        Show
        Cheolsoo Park added a comment - Attached is a patch that changes the name of the method in PigAvroDatumWriter from resolveUnion() to resolveUnionSchema() to avoid the conflict. I tested against Avro 1.5.3, 1.6.3, and 1.7.1. Thanks!
        Cheolsoo Park made changes -
        Field Original Value New Value
        Attachment PIG-2854.patch [ 12538842 ]
        Cheolsoo Park made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Cheolsoo Park made changes -
        Status In Progress [ 3 ] Patch Available [ 10002 ]
        Hide
        Santhosh Srinivasan added a comment -

        +1 looks good. I will commit it after running the tests.

        Show
        Santhosh Srinivasan added a comment - +1 looks good. I will commit it after running the tests.
        Hide
        Santhosh Srinivasan added a comment -

        The patch has been committed to trunk. I tested the changes for both Hadoop 20 and Hadoop 23. There are unrelated failure in TestDBStorage (20 and 23) and TestMultiStorage (23)

        Thanks Cheolsoo!

        Show
        Santhosh Srinivasan added a comment - The patch has been committed to trunk. I tested the changes for both Hadoop 20 and Hadoop 23. There are unrelated failure in TestDBStorage (20 and 23) and TestMultiStorage (23) Thanks Cheolsoo!
        Santhosh Srinivasan made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Cheolsoo Park made changes -
        Fix Version/s 0.11 [ 12318878 ]
        Bill Graham made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open In Progress In Progress
        3m 53s 1 Cheolsoo Park 01/Aug/12 23:17
        In Progress In Progress Patch Available Patch Available
        7s 1 Cheolsoo Park 01/Aug/12 23:17
        Patch Available Patch Available Resolved Resolved
        4h 27m 1 Santhosh Srinivasan 02/Aug/12 03:44
        Resolved Resolved Closed Closed
        204d 2h 9m 1 Bill Graham 22/Feb/13 04:53

          People

          • Assignee:
            Cheolsoo Park
            Reporter:
            Cheolsoo Park
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development