[FLINK-21132] BoundedOneInput.endInput is called when taking synchronous savepoint - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.10.2, 1.10.3, 1.11.3, 1.12.1
Fix Version/s: 1.11.4, 1.12.2, 1.13.0
Component/s: Runtime / Task
Labels:
- pull-request-available

Release Note:
endInput() is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is stopping with savepoint.

Description

Elkhan Dadashov reported on project iceberg that BoundedOneInput.endInput was called when stopping job with savepoint.

I think it is a bug of Flink and was introduced in ~~FLINK-14230~~. The changes rely on StreamTask.afterInvoke and OperatorChain.closeOperators will only be invoked after end of input. But that is not true long before after FLIP-34: Terminate/Suspend Job with Savepoint. Task could enter state called finished after synchronous savepoint, that is an expected job suspension and stopping.

Haibo Sun Piotr Nowojski [~roman_khachatryan] Could you help confirm this ?

For full context, see apache/iceberg#2033. I have pushed branch synchronous-savepoint-conflict-with-bounded-end-input-case in my repository. Test case SavepointITCase.testStopSavepointWithBoundedInput failed due to BoundedOneInput.endInput called.

I am also aware of FLIP-147: Support Checkpoints After Tasks Finished, maybe the three should align on what finished means exactly. Kostas Kloudas Chesnay Schepler Yun Gao