[FLINK-16913] ReadableConfigToConfigurationAdapter#getEnum throws UnsupportedOperationException

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.1, 1.11.0
• Component/s: Runtime / Configuration
• Labels: pull-request-available

Release Note

Starting from 1.11 the StateBackendFactory#createFromConfig interface now takes ReadableConfig instead of Configuration. A configuration class is still a valid argument to that method, as it implements the ReadableConfig interface. Implementors of custom StateBackends should adjust their implementations.

• Assignee: Dawid Wysakowicz
• Reporter: Canbin Zheng
• Created: 01/Apr/20 08:50
• Updated: 07/Apr/20 14:14
• Resolved: 07/Apr/20 11:54

Description

Steps to reproduce the issue:

1. Set flink-conf.yaml
   • state.backend: rocksdb
   • state.checkpoints.dir: hdfs:///flink-checkpoints
   • state.savepoints.dir: hdfs:///flink-checkpoints
2. Start a Kubernetes session cluster
3. Submit a job to the session cluster, unfortunately a UnsupportedOperationException occurs.

 The program finished with the following exception:org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: The adapter does not support this method
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
	at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
	at org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:143)
	at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:659)
	at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
	at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:890)
	at org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:963)
	at org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
	at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:963)
Caused by: java.lang.UnsupportedOperationException: The adapter does not support this method
	at org.apache.flink.configuration.ReadableConfigToConfigurationAdapter.getEnum(ReadableConfigToConfigurationAdapter.java:258)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.<init>(RocksDBStateBackend.java:336)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackend.configure(RocksDBStateBackend.java:394)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:47)
	at org.apache.flink.contrib.streaming.state.RocksDBStateBackendFactory.createFromConfig(RocksDBStateBackendFactory.java:32)
	at org.apache.flink.runtime.state.StateBackendLoader.loadStateBackendFromConfig(StateBackendLoader.java:154)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.loadStateBackend(StreamExecutionEnvironment.java:792)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.configure(StreamExecutionEnvironment.java:761)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.<init>(StreamExecutionEnvironment.java:217)
	at org.apache.flink.client.program.StreamContextEnvironment.<init>(StreamContextEnvironment.java:53)
	at org.apache.flink.client.program.StreamContextEnvironment.lambda$setAsContext$2(StreamContextEnvironment.java:103)
	at java.util.Optional.map(Optional.java:215)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.getExecutionEnvironment(StreamExecutionEnvironment.java:1882)
	at org.apache.flink.streaming.examples.socket.SocketWindowWordCount.main(SocketWindowWordCount.java:62)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
	... 8 more

I am wondering why we introduce ReadableConfigToConfigurationAdapter to wrap the Configuration but leave many of the getter methods in it to throw UnsupportedOperationException that causes potential problems.

Issue Links

• links to: GitHub Pull Request #11622
• links to: GitHub Pull Request #11637