Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
1.12.1
-
None
-
None
-
18.04.1-Ubuntu
Description
ConvertRecord processor with settings
Record Reader: JsonTreeReader (with Infer Schema)
Record Writer: AvroRecordSetWriter (with Embed Avro Schema)
change non schema field names avro (e.g. with "-", "@") to fields with underscore, but with NULL values.
Example below.
Input JSON:
{ "id":"6e07e04d-d644-bf76-f30f-ce648da478e0", "type":"agreement", "attributes":{ "code":"16-00-04332-20-01", "origin-code":"07-19-305", "service-type":"07", "service-name":"Leczenie stomatologiczne", "amount":112500.0, "updated-at":"2019-12-13T08:39:20", "provider-code":"160004332", "provider-name":"ISP STOMATOLOGICZNA JANUSZ KUBRAK", "provider-place":"SZCZECIN", "year":2020, "branch":"16" }, "links":{ "related":"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json" } }
Output Avro data:
[ { "id":"6e07e04d-d644-bf76-f30f-ce648da478e0", "type":"agreement", "attributes":{ "code":"16-00-04332-20-01", "origin_code":null, "service_type":null, "service_name":null, "amount":112500.0, "updated_at":null, "provider_code":null, "provider_name":null, "provider_place":null, "year":2020, "branch":"16" }, "links":{ "related":"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json" } } ]
Output Avro schema:
{ "type":"record", "name":"nifiRecord", "namespace":"org.apache.nifi", "fields":[ { "name":"id", "type":[ "null", "string" ] }, { "name":"type", "type":[ "null", "string" ] }, { "name":"attributes", "type":[ "null", { "type":"record", "name":"attributesType", "fields":[ { "name":"code", "type":[ "null", "string" ] }, { "name":"origin_code", "type":[ "null", "string" ], "aliases":[ "origin-code" ] }, { "name":"service_type", "type":[ "null", "string" ], "aliases":[ "service-type" ] }, { "name":"service_name", "type":[ "null", "string" ], "aliases":[ "service-name" ] }, { "name":"amount", "type":[ "null", "double" ] }, { "name":"updated_at", "type":[ "null", "string" ], "aliases":[ "updated-at" ] }, { "name":"provider_code", "type":[ "null", "string" ], "aliases":[ "provider-code" ] }, { "name":"provider_name", "type":[ "null", "string" ], "aliases":[ "provider-name" ] }, { "name":"provider_place", "type":[ "null", "string" ], "aliases":[ "provider-place" ] }, { "name":"year", "type":[ "null", "long" ] }, { "name":"branch", "type":[ "null", "string" ] } ] } ] }, { "name":"links", "type":[ "null", { "type":"record", "name":"linksType", "fields":[ { "name":"related", "type":[ "null", "string" ] } ] } ] } ] }
Workaround: for "static" JSONs - ReplaceTextWithMapping processor before ConvertRecord.