Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Not A Problem
-
1.10.0
-
None
-
None
Description
In Blink planner, we use ttl state to clean expired data by rocksdb for regular joins. The StateTtlConfig is:
StateTtlConfig .newBuilder(Time.milliseconds(retentionTime)) .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite) .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp) .build();
I think we should use StateTtlConfig.StateVisibility.NeverReturnExpired to prevent to join expired data.
BTW, Only regular joins use rocksdb filter to clean expired data, should we change timer to rocksdb filter in other sql operator such as TemporalTableJoin?