[FLINK-21132] BoundedOneInput.endInput is called when taking synchronous savepoint - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 1.10.2, 1.10.3, 1.11.3, 1.12.1
Fix Version/s: 1.11.4, 1.12.2, 1.13.0
Component/s: Runtime / Task
Labels:
- pull-request-available

Release Note:
endInput() is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is stopping with savepoint.

Description

Elkhan Dadashov reported on project iceberg that BoundedOneInput.endInput was called when stopping job with savepoint.

I think it is a bug of Flink and was introduced in ~~FLINK-14230~~. The changes rely on StreamTask.afterInvoke and OperatorChain.closeOperators will only be invoked after end of input. But that is not true long before after FLIP-34: Terminate/Suspend Job with Savepoint. Task could enter state called finished after synchronous savepoint, that is an expected job suspension and stopping.

Haibo Sun Piotr Nowojski Roman Khachatryan Could you help confirm this ?

For full context, see apache/iceberg#2033. I have pushed branch synchronous-savepoint-conflict-with-bounded-end-input-case in my repository. Test case SavepointITCase.testStopSavepointWithBoundedInput failed due to BoundedOneInput.endInput called.

I am also aware of FLIP-147: Support Checkpoints After Tasks Finished, maybe the three should align on what finished means exactly. Kostas Kloudas Chesnay Schepler Yun Gao