Pig
  1. Pig
  2. PIG-3297

Avro files with stringType set to String cannot be read by the AvroStorage LoadFunc

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.11.1
    • Fix Version/s: 0.12.0
    • Component/s: piggybank
    • Labels:
      None
    • Release Note:
      Hide
      Read Avro files that have string fields that were written with avro.java.string = String
      Show
      Read Avro files that have string fields that were written with avro.java.string = String

      Description

      When an Avro file is created there exists the option to set the "String Type" to a different class than the default Utf8.
      A very common situation is that the "String Type" is set to the default String class.

      When trying to read such an Avro file in Pig using the AvroStorage LoadFunc from the included piggybank this gives the following Exception:

      Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to org.apache.avro.util.Utf8
      at org.apache.pig.piggybank.storage.avro.PigAvroDatumReader.readString(PigAvroDatumReader.java:154)
      at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:150)

      1. test_record.avro
        0.2 kB
        Niels Basjes
      2. PIG-3297-v2-20130521-Unittest.patch
        11 kB
        Niels Basjes
      3. PIG-3297-v2-20130521-The-fix.patch
        0.8 kB
        Niels Basjes
      4. PIG-3297-1.patch
        5 kB
        Niels Basjes

        Issue Links

          Activity

            People

            • Assignee:
              Niels Basjes
              Reporter:
              Niels Basjes
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development