[FLINK-32476] Support configuring object-reuse for internal operators

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Runtime / Task
• Labels: auto-deprioritized-major pull-request-available
• Assignee: Unassigned
• Reporter: Xuannan Su
• Created: 29/Jun/23 02:32
• Updated: 26/Sep/23 01:53

Description

Currently, object reuse is disabled by default for streaming jobs in order to prevent unexpected behavior. Object reuse becomes problematic when the upstream operator stores its output while the downstream operator modifies the input.

However, many operators implemented by Flink, such as Flink SQL operators, do not modify the input. This implies that it is safe to reuse the input object in such cases. Therefore, we intend to enable object reuse specifically for operators that do not modify the input.

As the first step, we will focus on the operators implemented within Flink. We will create the FLIP to introduce the API that allows user-defined operators to enable object reuse in the future.

Issue Links

• links to: GitHub Pull Request #22897