-
Type:
Bug
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: 1.10.2, 1.10.3, 1.11.3, 1.12.1
-
Component/s: Runtime / Task
-
Labels:
-
Release Note:endInput() is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is stopping with savepoint.
Elkhan Dadashov reported on project iceberg that BoundedOneInput.endInput was called when stopping job with savepoint.
I think it is a bug of Flink and was introduced in FLINK-14230. The changes rely on StreamTask.afterInvoke and OperatorChain.closeOperators will only be invoked after end of input. But that is not true long before after FLIP-34: Terminate/Suspend Job with Savepoint. Task could enter state called finished after synchronous savepoint, that is an expected job suspension and stopping.
Haibo Sun Piotr Nowojski Roman Khachatryan Could you help confirm this ?
For full context, see apache/iceberg#2033. I have pushed branch synchronous-savepoint-conflict-with-bounded-end-input-case in my repository. Test case SavepointITCase.testStopSavepointWithBoundedInput failed due to BoundedOneInput.endInput called.
I am also aware of FLIP-147: Support Checkpoints After Tasks Finished, maybe the three should align on what finished means exactly. Kostas Kloudas Chesnay Schepler Yun Gao
- causes
-
FLINK-21453 BoundedOneInput.endInput is NOT called when doing stop with savepoint WITH drain
-
- Closed
-
- links to