Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21132

BoundedOneInput.endInput is called when taking synchronous savepoint

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Release Note:
      endInput() is not called anymore (on BoundedOneInput and BoundedMultiInput) when the job is stopping with savepoint.

      Description

      Elkhan Dadashov reported on project iceberg that BoundedOneInput.endInput was called when stopping job with savepoint.

      I think it is a bug of Flink and was introduced in FLINK-14230. The changes rely on StreamTask.afterInvoke and OperatorChain.closeOperators will only be invoked after end of input. But that is not true long before after FLIP-34: Terminate/Suspend Job with Savepoint. Task could enter state called finished after synchronous savepoint, that is an expected job suspension and stopping.

      Haibo Sun Piotr Nowojski [~roman_khachatryan] Could you help confirm this ?

      For full context, see apache/iceberg#2033. I have pushed branch synchronous-savepoint-conflict-with-bounded-end-input-case in my repository. Test case SavepointITCase.testStopSavepointWithBoundedInput failed due to BoundedOneInput.endInput called.

      I am also aware of FLIP-147: Support Checkpoints After Tasks Finished, maybe the three should align on what finished means exactly. Kostas Kloudas Chesnay Schepler Yun Gao

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              roman Roman Khachatryan
              Reporter:
              kezhuw Kezhu Wang

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment