Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21132

BoundedOneInput.endInput is called when taking synchronous savepoint

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Release Note:
      endInput() is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is stopping with savepoint.

      Description

      Elkhan Dadashov reported on project iceberg that BoundedOneInput.endInput was called when stopping job with savepoint.

      I think it is a bug of Flink and was introduced in FLINK-14230. The changes rely on StreamTask.afterInvoke and OperatorChain.closeOperators will only be invoked after end of input. But that is not true long before after FLIP-34: Terminate/Suspend Job with Savepoint. Task could enter state called finished after synchronous savepoint, that is an expected job suspension and stopping.

      Haibo Sun Piotr Nowojski Roman Khachatryan Could you help confirm this ?

      For full context, see apache/iceberg#2033. I have pushed branch synchronous-savepoint-conflict-with-bounded-end-input-case in my repository. Test case SavepointITCase.testStopSavepointWithBoundedInput failed due to BoundedOneInput.endInput called.

      I am also aware of FLIP-147: Support Checkpoints After Tasks Finished, maybe the three should align on what finished means exactly. Kostas Kloudas Chesnay Schepler Yun Gao

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              roman_khachatryan Roman Khachatryan
              Reporter:
              kezhuw Kezhu Wang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment