[FLINK-30498] frocksdb throw an UnsatisfiedLinkError

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 1.14.6
• Component/s: Runtime / State Backends
• Assignee: Unassigned
• Reporter: tanjialiang
• Created: 23/Dec/22 10:32
• Updated: 27/Dec/22 06:05
• Resolved: 23/Dec/22 11:03

Description

Using an simple TopN SQL with table.exec.state.ttl options, statebackend use rocksdb

SET table.exec.state.ttl = 1min;

CREATE TABLE `student` (
  id int,
  name string,
  update_time bigint
) WITH (
  'connector' = 'datagen',
  'fields.id.kind' = 'random',
  'fields.id.min' = '0',
  'fields.id.max' = '100'
);

CREATE TABLE `blackhole` (
  id int,
  name string,
  update_time bigint
) WITH (
  'connector' = 'blackhole'
);

INSERT INTO blackhole
SELECT 
    id, 
    name,
    update_time
FROM (
  SELECT
      id, 
      name,
      update_time,
      ROW_NUMBER() OVER (PARTITION BY id ORDER BY update_time DESC) AS row_num 
  FROM student
) AS t
WHERE t.row_num=1; 

It will throw an error from frocksdb

java.lang.UnsatisfiedLinkError: org.rocksdb.FlinkCompactionFilter.createNewFlinkCompactionFilterConfigHolder()J
    at org.rocksdb.FlinkCompactionFilter.createNewFlinkCompactionFilterConfigHolder(Native Method) ~[flink-dist_2.12-1.14.2.jar:?]
    at org.rocksdb.FlinkCompactionFilter.access$000(FlinkCompactionFilter.java:14) ~[flink-dist_2.12-1.14.2.jar:?]
    at org.rocksdb.FlinkCompactionFilter$ConfigHolder.<init>(FlinkCompactionFilter.java:115) ~[flink-dist_2.12-1.14.2.jar:?]
    at org.rocksdb.FlinkCompactionFilter$FlinkCompactionFilterFactory.<init>(FlinkCompactionFilter.java:142) ~[flink-dist_2.12-1.14.2.jar:?]
    at org.apache.flink.contrib.streaming.state.ttl.RocksDbTtlCompactFiltersManager.createAndSetCompactFilterFactory(RocksDbTtlCompactFiltersManager.java:84) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.contrib.streaming.state.ttl.RocksDbTtlCompactFiltersManager.setAndRegisterCompactFilterIfStateTtl(RocksDbTtlCompactFiltersManager.java:74) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.createColumnFamilyDescriptor(RocksDBOperationUtils.java:157) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.contrib.streaming.state.RocksDBOperationUtils.createStateInfo(RocksDBOperationUtils.java:134) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.tryRegisterKvStateInformation(RocksDBKeyedStateBackend.java:643) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.createInternalState(RocksDBKeyedStateBackend.java:837) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.ttl.TtlStateFactory.createTtlStateContext(TtlStateFactory.java:225) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.ttl.TtlStateFactory.createValueState(TtlStateFactory.java:148) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.ttl.TtlStateFactory.createState(TtlStateFactory.java:132) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.ttl.TtlStateFactory.createStateAndWrapWithTtlIfEnabled(TtlStateFactory.java:72) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.AbstractKeyedStateBackend.getOrCreateKeyedState(AbstractKeyedStateBackend.java:302) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.AbstractKeyedStateBackend.getPartitionedState(AbstractKeyedStateBackend.java:353) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.DefaultKeyedStateStore.getPartitionedState(DefaultKeyedStateStore.java:115) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.state.DefaultKeyedStateStore.getState(DefaultKeyedStateStore.java:60) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.api.operators.StreamingRuntimeContext.getState(StreamingRuntimeContext.java:203) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.table.runtime.operators.rank.FastTop1Function.open(FastTop1Function.java:114) ~[flink-table_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:100) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.api.operators.KeyedProcessOperator.open(KeyedProcessOperator.java:55) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:110) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:711) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$1.call(StreamTaskActionExecutor.java:55) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:687) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:654) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:958) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:927) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:766) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:575) ~[flink-dist_2.12-1.14.2.jar:1.14.2]
    at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_302] 

 

I try in kubernetes application mode, it will throw an error just in 1.14, not found in 1.15.3 and 1.16.0. And i try standalone mode in 1.14, error not found.

It is any change in Dockerfile between 1.14 and 1.15?