[FLINK-25002] Setup required --add-opens/--add-exports

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.18.0
• Component/s: Build System, Build System / CI, Documentation, Tests
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Chesnay Schepler
• Created: 22/Nov/21 14:01
• Updated: 16/Jun/23 17:02
• Resolved: 16/Jun/23 17:02

Description

Java 17 actually enforces the encapsulation of the JDK (opposed to Java 11 which just printed warnings), requiring us to explicitly open/export any package that we access illegally.

The following is a list of opens/exports that I needed to get most tests to pass, also with some comments which component needed them. Overall the ClosureCleaner and FieldSerializer result in the most offenses, as they try to access private fields.

These properties need to be set for all JVMs in which we run Flink, including surefire forks, other tests processes (TestJvmProcess/TestProcessBuilder/Yarn) and the distribution.
This needs some thought on how we can share this list across poms (surefire), code (test processes / yarn) and the configuration (distribution).

<surefire.module.config> <!--
	-->--add-exports java.base/sun.net.util=ALL-UNNAMED <!--
	required by JmxServer
	-->--add-exports java.rmi/sun.rmi.registry=ALL-UNNAMED <!--
	-->--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED <!--
	-->--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED <!--
	-->--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED <!--
	-->--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED <!--
	-->--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED <!--
	HadoopUtilsTest
	-->--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED <!--
	-->--add-opens java.base/java.lang=ALL-UNNAMED <!--
	RocksDBAsyncSnapshotTest Whitebox SerializedThrowable
	-->--add-opens java.base/java.lang.invoke=ALL-UNNAMED <!--
	RoundingMode
	-->--add-opens java.base/java.math=ALL-UNNAMED <!--
	-->--add-opens java.base/java.net=ALL-UNNAMED <!--
	InitOutputPathTest
	-->--add-opens java.base/java.io=ALL-UNNAMED <!--
	InitOutputPathTest
	-->--add-opens java.base/java.lang.reflect=ALL-UNNAMED <!--
	ClosureCleaner Timestamp
	-->--add-opens java.sql/java.sql=ALL-UNNAMED <!--
	lz4 Buffer
	-->--add-opens java.base/java.nio=ALL-UNNAMED <!--
	ClosureCleaner SimpleDateFormat
	-->--add-opens java.base/java.text=ALL-UNNAMED <!--
	FieldSerializer LocalDate
	-->--add-opens java.base/java.time=ALL-UNNAMED <!--
	ExpressionKeysTest
	-->--add-opens java.base/java.util=ALL-UNNAMED <!--
	AsynchronousFileIOChannelTest
	-->--add-opens java.base/java.util.concurrent=ALL-UNNAMED <!--
	Kryo FieldSerializer
	-->--add-opens java.base/java.util.concurrent.atomic=ALL-UNNAMED <!--
	ClosureCleaner ReentrantLock
	-->--add-opens java.base/java.util.concurrent.locks=ALL-UNNAMED <!--
	Whitebox ReferencePipeline$Head
	-->--add-opens java.base/java.util.stream=ALL-UNNAMED <!--
	ClosureCleaner ZoneInfo
	-->--add-opens java.base/sun.util.calendar=ALL-UNNAMED
</surefire.module.config>

Additionally, the following JVM arguments must be supplied when running Maven:

export MAVEN_OPTS="\
--add-exports jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED \
--add-exports jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED \
--add-exports java.security.jgss/sun.security.krb5=ALL-UNNAMED"

Issue Links

• Testing discovered: FLINK-32366 YarnConfigurationITCase.testFlinkContainerMemory fails with ConnectException (msg: Connection refused)
• links to: GitHub Pull Request #22794