[FLINK-15938] Idle state not cleaned in StreamingJoinOperator and StreamingSemiAntiJoinOperator

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.9.2
• Fix Version/s: 1.9.3
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Benchao Li
• Reporter: Benchao Li
• Created: 06/Feb/20 13:04
• Updated: 12/Feb/20 08:11
• Resolved: 12/Feb/20 08:11

Description

return StateTtlConfig
   .newBuilder(Time.milliseconds(retentionTime))
   .setUpdateType(StateTtlConfig.UpdateType.OnCreateAndWrite)
   .setStateVisibility(StateTtlConfig.StateVisibility.ReturnExpiredIfNotCleanedUp)
   .build();

StateTtl is constructed in above code for `StreamingJoinOperator` and `StreamingSemiAntiJoinOperator`.

However, as stated in https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/state/state.html#cleanup-of-expired-state , the state will be cleaned only when it's read which means the state will not be cleaned enless we read it.

Issue Links

• links to: GitHub Pull Request #11063