[NIFI-7996] Conversion with ConvertRecord to avro results in invalid date

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Assignee: Denes Arvay
• Reporter: Denes Arvay
• Created: 11/Nov/20 11:28
• Updated: 25/Jan/21 23:08
• Resolved: 16/Nov/20 16:41

Description

Converting a date field to avro using ConvertRecord results to invalid value if the system's timezone's offset is negative.

System timezone: EST (UTC-5)

Input json:

{ "SomeLocalDate": "20170411" }

Avro schema:

{ "fields": [ {
       "name": "SomeLocalDate",
       "type": [ "null", { "logicalType": "date", "type": "int" } ]
  }],
 "name": "DateTest",
 "namespace": "org.apache.nifi",
 "type": "record"
}

Result:

$ avro-tools tojson ./est-invalid.avro
{"SomeLocalDate":{"int":17266}}

In this case the days between 1970.01.01. and 2017.04.11. is stored in the SomeLocalDate field (see [1]), it should be 17267 ([2]).

After investigating the issue it seems that even though 20170411 is parsed to 2017-04-11T00:00:00 UTC it is converted to the system timezone (2017-04-10T19:00:00 EST) later and its date value is used.

[1] https://avro.apache.org/docs/1.8.0/spec.html#Date
[2] https://www.timeanddate.com/date/durationresult.html?d1=1&m1=1&y1=1970&d2=11&m2=04&y2=2017
 

Issue Links

• is related to: NIFI-8023 Test failure after NIFI-7996 commit if system default TZ is CET
• links to: GitHub Pull Request #4666