[FLINK-11193] Rocksdb timer service factory configuration option is not settable per job

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.6.2, 1.6.4, 1.7.0
• Fix Version/s: 1.8.4, 1.9.3, 1.10.1, 1.11.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available usability
• Assignee: WenJun Min
• Reporter: Fan weiwen
• Created: 18/Dec/18 16:01
• Updated: 08/Dec/21 08:50
• Resolved: 04/Mar/20 09:57

Description

set state backends is rocksdb 

set 'state.backend.rocksdb.timer-service.factory' by  Configuration

RocksDBStateBackend backend = new RocksDBStateBackend(checkpointPath,true);
Configuration configuration = new Configuration();
configuration.setString(
   RocksDBOptions.TIMER_SERVICE_FACTORY,
   RocksDBStateBackend.PriorityQueueStateType.ROCKSDB.toString());
backend = backend.configure(configuration);

then submit the job by client

but  the job runtime the 'TIMER_SERVICE_FACTORY'  Covered  by jobmaster

jobmaster code

org.apache.flink.runtime.state.StateBackendLoader

backend = ((ConfigurableStateBackend) fromApplication).configure(config);

jobmaster  buildGraph  config is from flink-conf.yaml

Final execution of the same method

org.apache.flink.contrib.streaming.state.RocksDBStateBackend

final String priorityQueueTypeString = config.getString(TIMER_SERVICE_FACTORY);

this.priorityQueueStateType = priorityQueueTypeString.length() > 0 ?
   PriorityQueueStateType.valueOf(priorityQueueTypeString.toUpperCase()) : original.priorityQueueStateType;

the problem is  config.getString(TIMER_SERVICE_FACTORY)   always have value regardless of configuration

so my customize config is Covered by jobmaster

 

Issue Links

• fixes: FLINK-10372 There is no API to configure the timer state backend
• is duplicated by: FLINK-11192 rockdb customize configuration Covered jobmaster
• is duplicated by: FLINK-10098 Programmatically select timer storage backend
• links to: GitHub Pull Request #7330
• links to: GitHub Pull Request #8479