[AVRO-3810] Incorrect JSON generated by avro::jsonEncoder from avro::GenericDatum when schema contains as last value a record with 0 field

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: c++

Environment

Windows, 64bit, compiled with msvc

Language

C++

• Assignee: Thiruvalluvan M. G.
• Reporter: Jerome Delrieu
• Created: 21/Jul/23 13:33
• Updated: 06/Apr/24 03:10
• Resolved: 06/Apr/24 03:10

Description

When the Schema has a record with 0 field as its last value, the avro::jsonEncoder doesn't generate the last characters in the JSON string.

The obtained JSON string is malformed and cannot be parsed with a standard JSON parser.

Note : the avro::jsonDecoder accept this string whereas it is not a valid JSON syntax

small C++ example:

const char *sSchema = "{\"type\": \"record\",\"name\" : \"Event\",\"namespace\" : \"test\",\"doc\" : \"\", \"fields\" : "   "[ { \"name\": \"Event\", \"type\" : [ {  \"type\": \"record\",  \"name\" : \"ABEvent\", \"doc\" : \"unused\","   "\"fields\" : [ ] } ] } ] }";

avro::ValidSchema schema(avro::compileJsonSchemaFromString(sSchema)); avro::GenericDatum datumDecode(schema); avro::DecoderPtr decoder = avro::binaryDecoder();

// only one byte, 0, as source for decoding uint8_t binarySource = 0; std::unique_ptr<avro::InputStream> in = avro::memoryInputStream((uint8_t *)&binarySource, 1); decoder->init(*in); avro::decode(*decoder, datumDecode);

std::unique_ptr<avro::OutputStream> out = avro::memoryOutputStream();

std::shared_ptr<avro::Encoder> encoder = avro::jsonEncoder(schema);

encoder->init(*out); avro::encode(*encoder, datumDecode); encoder->flush();

const std::shared_ptr<std::vector<uint8_t> > &buffer = avro::snapshot(*out.get());

std::string result(reinterpret_cast<char *>(buffer.get()>data()), buffer.get()>size());

 

Example Schema:

Schema:
{
  "type": "record",
  "name": "Event",
  "namespace": "test",
  "doc": "",
  "fields": [
    {
      "name": "Event",
      "type": [
       

{           "type": "record",           "name": "ABEvent",           "doc": "unused",           "fields": [                        ]         }

      ]
    }
  ]
}

 

Binary source : 1 byte 00

Result:

{"Event":{"test.ABEvent":

 

 

A small patch to lang\c++\impl\parsing\Symbol.hh can solve this issue
Before:
    void processImplicitActions() {
        for (; {
            Symbol &s = parsingStack.top();
            if (s.isImplicitAction())

{                 handler_.handle(s);                 parsingStack.pop();             } else if (s.kind() == Symbol::Kind::SkipStart) {                 parsingStack.pop();                 skip(*decoder_);             } else {                 break;             }
        }
    }

After:

    void processImplicitActions() {
        for (; {
            Symbol &s = parsingStack.top();
            if (s.isImplicitAction()) {                 handler_.handle(s);                 parsingStack.pop();             }

else if (s.kind() == Symbol::Kind::SkipStart)

{                 parsingStack.pop();                 skip(*decoder_);             }

else if (s.kind() == Symbol::Kind::Indirect)

{                 parsingStack.pop();                 parsingStack.push(Symbol::recordEndSymbol());                 parsingStack.push(Symbol::recordStartSymbol());             }

else

{                 break;             }

        }
    }