Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3297

Avro files with stringType set to String cannot be read by the AvroStorage LoadFunc

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.1
    • Fix Version/s: 0.12.0
    • Component/s: piggybank
    • Labels:
      None
    • Release Note:
      Hide
      Read Avro files that have string fields that were written with avro.java.string = String
      Show
      Read Avro files that have string fields that were written with avro.java.string = String

      Description

      When an Avro file is created there exists the option to set the "String Type" to a different class than the default Utf8.
      A very common situation is that the "String Type" is set to the default String class.

      When trying to read such an Avro file in Pig using the AvroStorage LoadFunc from the included piggybank this gives the following Exception:

      Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.avro.util.Utf8
      at org.apache.pig.piggybank.storage.avro.PigAvroDatumReader.readString(PigAvroDatumReader.java:154)
      at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)

        Attachments

        1. PIG-3297-1.patch
          5 kB
          Niels Basjes
        2. PIG-3297-v2-20130521-The-fix.patch
          0.8 kB
          Niels Basjes
        3. PIG-3297-v2-20130521-Unittest.patch
          11 kB
          Niels Basjes
        4. test_record.avro
          0.2 kB
          Niels Basjes

          Issue Links

            Activity

              People

              • Assignee:
                nielsbasjes Niels Basjes
                Reporter:
                nielsbasjes Niels Basjes
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: