[FLINK-33161] [benchmark] Java17 profile for benchmarks

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.18.1
• Component/s: Benchmarks
• Labels: pull-request-available
• Assignee: Zakelly Lan
• Reporter: Zakelly Lan
• Created: 27/Sep/23 03:17
• Updated: 09/Oct/23 02:08
• Resolved: 09/Oct/23 02:08

Description

Flink has supported java 17, however, the benchmark does not support running with java 17 out of the box. The runner complains:

Caused by: java.lang.RuntimeException: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @14899482
01:39:16 at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:69)
01:39:16 at org.apache.flink.api.java.typeutils.runtime.kryo.FlinkChillPackageRegistrar.registerSerializers(FlinkChillPackageRegistrar.java:67)
01:39:16 at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.getKryoInstance(KryoSerializer.java:513)
01:39:16 at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.checkKryoInitialized(KryoSerializer.java:522)
01:39:16 at org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:348)
01:39:16 at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:165)
01:39:16 at org.apache.flink.streaming.runtime.streamrecord.StreamElementSerializer.serialize(StreamElementSerializer.java:43)
01:39:16 at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:54)
01:39:16 at org.apache.flink.runtime.io.network.api.writer.RecordWriter.serializeRecord(RecordWriter.java:151)
01:39:16 at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:107)
01:39:16 at org.apache.flink.runtime.io.network.api.writer.ChannelSelectorRecordWriter.emit(ChannelSelectorRecordWriter.java:55)
01:39:16 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.pushToRecordWriter(RecordWriterOutput.java:134)
01:39:16 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collectAndCheckIfChained(RecordWriterOutput.java:114)
01:39:16 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:95)
01:39:16 at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:48)
01:39:16 at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:59)
01:39:16 at org.apache.flink.streaming.api.operators.CountingOutput.collect(CountingOutput.java:31)
01:39:16 at org.apache.flink.streaming.api.operators.StreamSourceContexts$ManualWatermarkContext.processAndCollect(StreamSourceContexts.java:425)
01:39:16 at org.apache.flink.streaming.api.operators.StreamSourceContexts$WatermarkContext.collect(StreamSourceContexts.java:520)
01:39:16 at org.apache.flink.streaming.api.operators.StreamSourceContexts$SwitchingOnClose.collect(StreamSourceContexts.java:110)
01:39:16 at org.apache.flink.benchmark.ContinuousFileReaderOperatorBenchmark$MockSourceFunction.run(ContinuousFileReaderOperatorBenchmark.java:101)
01:39:16 at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:114)
01:39:16 at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:71)
01:39:16 at org.apache.flink.streaming.runtime.tasks.SourceStreamTask$LegacySourceFunctionThread.run(SourceStreamTask.java:338)
01:39:16 Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field private final java.lang.Object[] java.util.Arrays$ArrayList.a accessible: module java.base does not "opens java.util" to unnamed module @14899482
01:39:16 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
01:39:16 at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
01:39:16 at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
01:39:16 at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
01:39:16 at com.twitter.chill.java.ArraysAsListSerializer.<init>(ArraysAsListSerializer.java:67)
01:39:16 ... 23 more
 

To resolve this, we should add a target profile for java 17 in pom.xml of benchmark project and provide some cli arg like "--add-opens java.base/java.util=ALL-UNNAMED" for the execution.

Issue Links

• is related to: FLINK-15736 Support Java 17 (LTS)
• links to: GitHub Pull Request #80