Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-29157

Clarify the contract between CompletedCheckpointStore and SharedStateRegistry

Details

    Description

      After FLINK-24611, CompletedCheckpointStore is required to call SharedStateRegistry.unregisterUnusedState() on checkpoint subsumption and shutdown.

      Although it's not clear whether CompletedCheckpointStore is internal there are in fact external implementations (which weren't updated accordingly).

       

      After FLINK-25872, CompletedCheckpointStore also must call checkpointsCleaner.cleanSubsumedCheckpoints.

       

      Another issue with a custom implementation was using different java objects for state for CheckpointStore and SharedStateRegistry (after FLINK-24086). 

       

      So it makes sense to:

      • clarify the contract (different in 1.15 and 1.16)
      • require using the same checkpoint objects by SharedStateRegistryFactory and CompletedCheckpointStore
      • mark the interface(s) as PublicEvolving

      Attachments

        Issue Links

          Activity

            Yanfei Lei Yanfei Lei added a comment - - edited

            > mark the interface(s) as PublicEvolving

            Since mark `CompletedCheckpointStore` would cause `CompletedCheckpoint` , `SharedStateRegistry`, `CheckpointsCleaner`, `StreamStateHandle`, `OperatorState`, `CompletedCheckpointStats`  ...  to change with it, and some of them should be internal. So don't mark `CompletedCheckpointStore` as PulicEvolving for now.

            Yanfei Lei Yanfei Lei added a comment - - edited > mark the interface(s) as PublicEvolving Since mark `CompletedCheckpointStore` would cause `CompletedCheckpoint` , `SharedStateRegistry`, `CheckpointsCleaner`, `StreamStateHandle`, `OperatorState`, `CompletedCheckpointStats`  ...  to change with it, and some of them should be internal. So don't mark `CompletedCheckpointStore` as PulicEvolving for now.
            klion26 Congxian Qiu added a comment - - edited

            merged into master 63767c5ed91642c67f97d9f16ff2b8955f9ae421

            1.16 be2bd93838548f7858baecf5e8beb469836081d5

            klion26 Congxian Qiu added a comment - - edited merged into master 63767c5ed91642c67f97d9f16ff2b8955f9ae421 1.16 be2bd93838548f7858baecf5e8beb469836081d5

            People

              Yanfei Lei Yanfei Lei
              roman Roman Khachatryan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: