Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-32070

FLIP-306 Unified File Merging Mechanism for Checkpoints

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Hide
      The unified file merging mechanism for checkpointing is introduced to Flink 1.20 as an MVP ("minimum viable product") feature, which allows scattered small checkpoint files to be written into larger files, reducing the number of file creations and file deletions and alleviating the pressure of file system metadata management raised by the file flooding problem during checkpoints. The mechanism can be enabled by setting `state.checkpoints.file-merging.enabled` to `true`. For more advanced options and principle behind this feature, please refer to the document of `Checkpointing`.
      Show
      The unified file merging mechanism for checkpointing is introduced to Flink 1.20 as an MVP ("minimum viable product") feature, which allows scattered small checkpoint files to be written into larger files, reducing the number of file creations and file deletions and alleviating the pressure of file system metadata management raised by the file flooding problem during checkpoints. The mechanism can be enabled by setting `state.checkpoints.file-merging.enabled` to `true`. For more advanced options and principle behind this feature, please refer to the document of `Checkpointing`.

    Description

      The FLIP: https://cwiki.apache.org/confluence/display/FLINK/FLIP-306%3A+Unified+File+Merging+Mechanism+for+Checkpoints

       

      The creation of multiple checkpoint files can lead to a 'file flood' problem, in which a large number of files are written to the checkpoint storage in a short amount of time. This can cause issues in large clusters with high workloads, such as the creation and deletion of many files increasing the amount of file meta modification on DFS, leading to single-machine hotspot issues for meta maintainers (e.g. NameNode in HDFS). Additionally, the performance of object storage (e.g. Amazon S3 and Alibaba OSS) can significantly decrease when listing objects, which is necessary for object name de-duplication before creating an object, further affecting the performance of directory manipulation in the file system's perspective of view (See hadoop-aws module documentation, section 'Warning #2: Directories are mimicked').

      While many solutions have been proposed for individual types of state files (e.g. FLINK-11937 for keyed state (RocksDB) and FLINK-26803 for channel state), the file flood problems from each type of checkpoint file are similar and lack systematic view and solution. Therefore, the goal of this FLIP is to establish a unified file merging mechanism to address the file flood problem during checkpoint creation for all types of state files, including keyed, non-keyed, channel, and changelog state. This will significantly improve the system stability and availability of fault tolerance in Flink.

      Attachments

        Issue Links

        1.
        Implement the snapshot manager for merged checkpoint files in TM Sub-task Closed Zakelly Lan Actions
        2.
        Implement file merging in snapshot Sub-task Closed Han Yin Actions
        3.
        Create and wire FileMergingSnapshotManager with TaskManagerServices Sub-task Closed Yanfei Lei Actions
        4.
        Delete merged files on checkpoint abort or subsumption Sub-task Resolved Zakelly Lan Actions
        5.
        Support file merging across checkpoints Sub-task Resolved Zakelly Lan Actions
        6.
        Report State handle of file merging directory to JM Sub-task Closed Yanfei Lei Actions
        7.
        Add file pool for concurrent file reusing Sub-task Resolved Hangxiang Yu Actions
        8.
        Implement shared state file merging Sub-task Closed Zakelly Lan Actions
        9.
        Implement private state file merging Sub-task Closed Yanfei Lei Actions
        10.
        Read/write checkpoint metadata of merged files Sub-task Resolved Hangxiang Yu Actions
        11.
        Register reused state handles to FileMergingSnapshotManager Sub-task Resolved Zakelly Lan Actions
        12.
        Restoration of FileMergingSnapshotManager Sub-task Resolved Jinzhong Li Actions
        13.
        Compatibility between file-merging on and off across job runs Sub-task Resolved Jinzhong Li Actions
        14.
        Documentation of checkpoint file-merging Sub-task Closed Yanfei Lei Actions
        15.
        Chinese translation of documentation of checkpoint file-merging Sub-task Closed Hangxiang Yu Actions
        16.
        Migrate current file merging of channel state into the file merging framework Sub-task Closed Yanfei Lei Actions
        17.
        Implement and migrate batch uploading in changelog files into the file merging framework Sub-task Open Hangxiang Yu Actions
        18.
        Space amplification statistics of file merging Sub-task Closed Yanfei Lei Actions
        19.
        Introduce file merging configuration Sub-task Closed Yanfei Lei Actions
        20.
        Re-uploading in state file-merging for space amplification control Sub-task Resolved Zakelly Lan Actions
        21.
        Do fast copy in best-effort during first checkpoint after restoration Sub-task Open Yanfei Lei Actions
        22.
        Python API for enabling and configuring file merging snapshot Sub-task Open Yanfei Lei Actions
        23.
        Add necessary metrics for file-merging Sub-task Closed Yanfei Lei Actions
        24.
        Integrate snapshot file-merging with existing IT cases Sub-task Closed Yanfei Lei Actions
        25.
        File merging manager is not properly notified about checkpoint Sub-task Resolved Zakelly Lan Actions
        26.
        Rebuild FileMergingSnapshotManager in failover Sub-task Resolved Zakelly Lan Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            zakelly Zakelly Lan
            zakelly Zakelly Lan

            Dates

              Created:
              Updated:

              Slack

                Issue deployment