Avro
  1. Avro
  2. AVRO-1066

ArrayIndexOutOfBoundsException in ParsingEncoder when trying to use a json encoder to serialize a deep object graph

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.6.3
    • Fix Version/s: 1.7.0
    • Component/s: None
    • Labels:
      None

      Description

      I have an avro record that can have an array of children records of the same type. If this structure gets to be too deep then I continually get an ArrayOutOfBoundsException in ParsingEncoder when trying to use a json encoder. This works fine when using a binary encoder. My schema looks something like this:

      {
      "name" : "MyRecord",
      "type" : "record",
      "fields" : [

      { "name" : "fooField", "type" : "int" }

      ,
      {
      "name" : "childRecords",
      "type" : [ "null",

      { "type" : "array", "items" : "MyRecord" }

      ]
      }
      ]
      }

      The code I'm using to capture a JSON string for debugging looks like this:

      ByteArrayOutputStream os = null;
      try

      { os = new ByteArrayOutputStream(2048); final Encoder jsonEncoder = EncoderFactory.get().jsonEncoder(MyRecord.SCHEMA$, os); final DatumWriter<MyRecord> datumWriter = new SpecificDatumWriter<MyRecord>(MyRecord.class); datumWriter.write(record, jsonEncoder); jsonEncoder.flush(); return new String(os.toByteArray(), Charset.defaultCharset()); }

      catch (IOException e)

      { return null; }

      finally {
      if (os != null) {
      try

      { os.close(); }

      catch (IOException e) {
      }
      }
      }

      The error I get looks like this:
      java.lang.ArrayIndexOutOfBoundsException: 10
      at org.apache.avro.io.ParsingEncoder.push(ParsingEncoder.java:55)
      at org.apache.avro.io.JsonEncoder.writeArrayStart(JsonEncoder.java:231)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:125)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.writeArray(GenericDatumWriter.java:129)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:105)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:65)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:57)
      <my line calling datumWriter.write>

      1. AVRO-1066-2.patch
        1 kB
        Thiruvalluvan M. G.
      2. AVRO-1066.patch
        4 kB
        Thiruvalluvan M. G.

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Daniel Lord
            Reporter:
            Daniel Lord
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development