[AVRO-2160] Json to Avro with non required value and union schema failing

• Type: Bug
• Status: Patch Available
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.8.2
• Fix Version/s: 1.12.0
• Component/s: java
• Labels: java pull-request-available
• Assignee: Christophe Le Saec
• Reporter: Lydie
• Created: 15/Mar/18 21:24
• Updated: 01/Feb/24 20:07

Description

I am trying to convert this string:

str str4

using this schema:

{"type":"record", "namespace":"foo","name":"Person","fields":[

{"name":"lastname","type": ["null","string"], "default":null}

,{"name":"firstname","type":"string"},

{"name":"age","type":["null","int"], "default":null}

]}

I get this error 

com.syapse.messagePublisher.publisher.AvroEncodeException: Expected field name not found: lastnamein{"firstname":"John","age":{"int":35}}at com.syapse.messagePublisher.publisher.AvroEncoder.convertJsonToAvro(AvroEncoder.java:78)

 

Although this should give me the correct syntax for a non required filed.

Note that it works for 

{"lastname":

{"string" : "Doe"}

,"firstname":"John","age":{"int":36}}

 

What am I missing ( using Abro 1.8.2)

here is my code:

 

public static byte[] convertJsonToAvro(byte[] data, String schemaStr) throws AvroEncodeException {

InputStream input = null;

DataFileWriter<GenericRecord> writer = null;

ByteArrayOutputStream output = null;

try {
Schema schema = new Schema.Parser().parse(schemaStr);
DatumReader<GenericRecord> reader = new GenericDatumReader<GenericRecord>(schema);
input = new ByteArrayInputStream(data);
DataInputStream din = new DataInputStream(input);
output = new ByteArrayOutputStream();
writer = new DataFileWriter<GenericRecord>(new GenericDatumWriter<GenericRecord>());
writer.create(schema, output);
Decoder decoder = DecoderFactory.get().jsonDecoder(schema, din);
GenericRecord datum = null;
while (true) {
try {
datum = reader.read(null, decoder);
} catch (EOFException eofe) {
break;
}
writer.append(datum);
}
writer.flush();
writer.close();
return output.toByteArray();
} catch (AvroTypeException e) {
throw new AvroEncodeException(e.getMessage() + "in" + new String(data));
} catch (IOException e1) {
throw new AvroEncodeException("Error decoding Json " + e1.getMessage());
} finally {
try {
input.close();
} catch (Exception e) {
}
}

}

 

Issue Links

• links to: GitHub Pull Request #1723
• links to: GitHub Pull Request #1723