[FLINK-35482] ParquetInt64TimestampReaderTest unit tests fail due to system timezone

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.20.0, 1.19.1
• Fix Version/s: 1.19.2, 1.20.1, 2.0-preview
• Component/s: Tests
• Labels: pull-request-available

Environment

on master commit 89c89d8522f779986f3f6f163d803e5d5f11ec62 (HEAD -> master, origin/master, origin/HEAD)

linux version: uname -r
6.9.1-arch1-1

checked with:

openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)

openjdk 17.0.5 2022-10-18
OpenJDK Runtime Environment Temurin-17.0.5+8 (build 17.0.5+8)

openjdk version "11.0.21" 2023-10-17 LTS
OpenJDK Runtime Environment Zulu11.68+17-CA (build 11.0.21+9-LTS)

openjdk version "1.8.0_382"
OpenJDK Runtime Environment (Temurin)(build 1.8.0_382-b05)

• Assignee: Unassigned
• Reporter: Rob Young
• Created: 30/May/24 00:56
• Updated: 02/Sep/24 09:35
• Resolved: 23/Aug/24 14:37

Description

To reproduce:

export TZ=Pacific/Auckland
./mvnw -Dfast -DskipTests -Dskip.npm=true install -pl :flink-parquet -am
./mvnw test -pl :flink-parquet

The parquet tests fail with:

[ERROR] Failures: 
[ERROR]   ParquetInt64TimestampReaderTest.testReadInt64TimestampMicros:46 expected:<2021-11-22T1[7]:50:20.000112> but was:<2021-11-22T1[8]:50:20.000112>
[ERROR]   ParquetInt64TimestampReaderTest.testReadInt64TimestampMillis:66 expected:<2021-11-22T1[7]:50:20> but was:<2021-11-22T1[8]:50:20>
[ERROR]   ParquetInt64TimestampReaderTest.testReadInt64TimestampNanos:78 expected:<2021-11-22T1[7]:50:20.000112233> but was:<2021-11-22T1[8]:50:20.000112233>

I think this is because the tests convert a LocalDateTime to epoch seconds using `OffsetDateTime.now().getOffset()` as the offset, but now's offset is different to what it would be at 2021-11-22T17:50:20.000112 NZST due to daylight savings.

Instead of using now's offset we could convert the localDateTime to a zonedDateTime using `localDateTime.atZone(ZoneId.systemDefault())`. If you're happy with that idea please assign me and I'll make a PR.

Another possible idea would be to set the user.timezone to GMT in the base argLine to use a consistent timezone for tests, which would be picked up by surefire and IDEA. But improving the tests feels like a better solution.

Thanks

 

Issue Links

• links to: GitHub Pull Request #25239
• links to: GitHub Pull Request #25241