Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-23883 (1.15) Follow up tasks and code improvements for FLIP-147
  3. FLINK-25105

Enables final checkpoint by default

    XMLWordPrintableJSON

Details

    • Hide
      In 1.15 we enabled the support of checkpoints after part of tasks finished by default, and make tasks waiting for the final checkpoint before exit to ensures all data get committed.

      However, it's worth noting that this change forces tasks to wait for one more checkpoint before exiting. In other words, this change will block the tasks until the next checkpoint get triggered and completed. If the checkpoint interval is long, the tasks' execution time would also be extended largely. In the worst case if the checkpoint interval is Long.MAX_VALUE, the tasks would be in fact blocked forever.

      More information about this feature and how to disable it could be found in https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpointing/#checkpointing-with-parts-of-the-graph-finished-beta.
      Show
      In 1.15 we enabled the support of checkpoints after part of tasks finished by default, and make tasks waiting for the final checkpoint before exit to ensures all data get committed. However, it's worth noting that this change forces tasks to wait for one more checkpoint before exiting. In other words, this change will block the tasks until the next checkpoint get triggered and completed. If the checkpoint interval is long, the tasks' execution time would also be extended largely. In the worst case if the checkpoint interval is Long.MAX_VALUE, the tasks would be in fact blocked forever. More information about this feature and how to disable it could be found in https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpointing/#checkpointing-with-parts-of-the-graph-finished-beta .

    Description

      We would fix all the conflict tests and enable the feature by default in this version.

      Attachments

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. FLINK-25556 Extra waiting of final checkpoint in benchmarks

          • Major - Major loss of function.
          • Closed
          links to

          Web Link GitHub Pull Request #18068

          Activity

            People

              gaoyunhaii Yun Gao
              gaoyunhaii Yun Gao
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: