[FLINK-32488] Introduce configuration to control ExecutionGraph cache in REST API

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.16.2, 1.17.1
• Fix Version/s: 1.20.0
• Component/s: Runtime / REST
• Labels: pull-request-available
• Assignee: Jufang He
• Reporter: Hong Liang Teoh
• Created: 29/Jun/23 15:27
• Updated: 11/Mar/24 12:43

Description

What

Currently, REST handlers that inherit from AbstractExecutionGraphHandler serve information derived from a cached ExecutionGraph.

This ExecutionGraph cache currently derives it's timeout from web.refresh-interval. The web.refresh-interval controls both the refresh rate of the Flink dashboard and the ExecutionGraph cache timeout. 

We should introduce a new configuration to control the ExecutionGraph cache, namely rest.cache.execution-graph.expiry.

Why

Sharing configuration between REST handler and Flink dashboard is a sign that we are coupling the two. 

Ideally, we want our REST API behaviour to independent of the Flink dashboard (e.g. supports programmatic access).

 

Mailing list discussion: https://lists.apache.org/thread/7o330hfyoqqkkrfhtvz3kp448jcspjrm

 

 

Issue Links

• relates to: FLINK-32898 Improvement of usability for Flink OLAP
• links to: GitHub Pull Request #22906
• links to: GitHub Pull Request #23387