Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-21352 FLIP-158: Generalized incremental checkpoints
  3. FLINK-21504

Enrich checkpoint barriers with previous checkpoint IDs

    XMLWordPrintableJSON

    Details

      Description

      As discussed in FLIP-151 thread both FLIP-151 and FLIP-158 would benefit from adding latest confirmed checkpoint ID to checkpoint barriers and RPC calls. More specifically, knowing the latest confirmed checkpoint at the time of performing a new one would allow to:

      • avoid otherwise unnecessary re-uploading of state changes
      • cleanup state changes uploaded to DFS and sent to JM but not used by the JM

      Besides that, adding earliest non-subsumed checkpoint ID would be another way to prevent FLINK-21351.

       
      cc: Stephan Ewen
       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              roman_khachatryan Roman Khachatryan
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: