Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14551

Unaligned checkpoints

    XMLWordPrintableJSON

    Details

      Description

      This is the umbrella issue for the feature of unaligned checkpoints. Refer to the FLIP-76 for more details.

       

       

        Attachments

          Issue Links

          1.
          Remove unnecessary interface method BufferProvider#requestBufferBlocking Sub-task Closed Zhijiang
          2.
          Implement rudimentary non-blocking network output Sub-task Resolved Zhijiang
          3.
          Introduce NetworkBufferPool#isAvailable() for non-blocking output Sub-task Closed Yingjie Cao
          4.
          Implement back-pressure monitor with non-blocking outputs Sub-task Closed Yingjie Cao
          5.
          Respect non-blocking output in StreamTask#processInput Sub-task Resolved Zhijiang
          6.
          Add API to persist channel state Sub-task Closed Roman Khachatryan
          7.
          Implement API to persist channel state: checkpointing metadata Sub-task Closed Roman Khachatryan
          8.
          Implement InputChannel state recovery for unaligned checkpoint Sub-task Closed Zhijiang
          9.
          Implement ResultPartition state recovery for unaligned checkpoint Sub-task Closed Zhijiang
          10.
          Build ResultSubpartitionInfo and InputChannelInfo in respective constructors Sub-task Closed Zhijiang
          11.
          Add basic CheckpointBarrierHandler for unaligned checkpoint Sub-task Closed Arvid Heise
          12.
          Implement API to persist channel state Sub-task Resolved Roman Khachatryan
          13.
          Add recovery to integration tests of unaligned checkpoints Sub-task Closed Arvid Heise  
          14.
          Persist spilled records of SpillingAdaptiveSpanningRecordDeserializer Sub-task Closed Roman Khachatryan  
          15.
          Enable unaligned checkpoints in tests by default Sub-task Closed Arvid Heise  
          16.
          Add checkpoint cancellation to Unaligner Sub-task Closed Roman Khachatryan
          17.
          Refactor the usages of ThreadSafeUnaligner#storeNewBuffers Sub-task Open Zhijiang  
          18.
          Schedule savepoint if max-inflight-checkpoints limit is reached instead of forcing (in UC mode) Sub-task Closed Roman Khachatryan  
          19.
          Implement aligned savepoint in UC mode Sub-task Closed Roman Khachatryan  
          20.
          Provide ChannelStateReader#hasStates for hints of reading channel states Sub-task Closed Zhijiang  
          21.
          Create integration or e2e test for out of order (savepoint) barriers Sub-task Open Unassigned  
          22.
          Support unaligned checkpoints for records spilled to files Sub-task Closed Roman Khachatryan  
          23.
          Add check for max concurrent checkpoints under UC mode Sub-task Closed Yuan Mei  
          24.
          Update documentation taking unaligned checkpoints into the account Sub-task Closed Arvid Heise  

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                zjwang Zhijiang
              • Votes:
                0 Vote for this issue
                Watchers:
                21 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 4h 20m
                  4h 20m