Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14551

Unaligned checkpoints

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

      Description

      This is the umbrella issue for the feature of unaligned checkpoints. Refer to the FLIP-76 for more details.

       

       

        Attachments

        Issue Links

        1.
        Remove unnecessary interface method BufferProvider#requestBufferBlocking Sub-task Closed Zhijiang Actions
        2.
        Implement rudimentary non-blocking network output Sub-task Closed Zhijiang Actions
        3.
        Introduce NetworkBufferPool#isAvailable() for non-blocking output Sub-task Closed Yingjie Cao Actions
        4.
        Implement back-pressure monitor with non-blocking outputs Sub-task Closed Yingjie Cao Actions
        5.
        Respect non-blocking output in StreamTask#processInput Sub-task Closed Zhijiang Actions
        6.
        Add API to persist channel state Sub-task Closed Roman Khachatryan Actions
        7.
        Implement API to persist channel state: checkpointing metadata Sub-task Closed Roman Khachatryan Actions
        8.
        Implement InputChannel state recovery for unaligned checkpoint Sub-task Closed Zhijiang Actions
        9.
        Implement ResultPartition state recovery for unaligned checkpoint Sub-task Closed Zhijiang Actions
        10.
        Build ResultSubpartitionInfo and InputChannelInfo in respective constructors Sub-task Closed Zhijiang Actions
        11.
        Add basic CheckpointBarrierHandler for unaligned checkpoint Sub-task Closed Arvid Heise Actions
        12.
        Implement API to persist channel state Sub-task Closed Roman Khachatryan Actions
        13.
        Add recovery to integration tests of unaligned checkpoints Sub-task Closed Arvid Heise   Actions
        14.
        Persist spilled records of SpillingAdaptiveSpanningRecordDeserializer Sub-task Closed Roman Khachatryan   Actions
        15.
        Enable unaligned checkpoints in tests by default Sub-task Closed Arvid Heise   Actions
        16.
        Add checkpoint cancellation to Unaligner Sub-task Closed Roman Khachatryan Actions
        17.
        Refactor the usages of ThreadSafeUnaligner#storeNewBuffers Sub-task Closed Zhijiang   Actions
        18.
        Schedule savepoint if max-inflight-checkpoints limit is reached instead of forcing (in UC mode) Sub-task Closed Roman Khachatryan   Actions
        19.
        Implement aligned savepoint in UC mode Sub-task Closed Roman Khachatryan   Actions
        20.
        Provide ChannelStateReader#hasStates for hints of reading channel states Sub-task Closed Zhijiang   Actions
        21.
        Support unaligned checkpoints for records spilled to files Sub-task Closed Roman Khachatryan   Actions
        22.
        Add check for max concurrent checkpoints under UC mode Sub-task Closed Yuan Mei   Actions
        23.
        Update documentation taking unaligned checkpoints into the account Sub-task Closed Arvid Heise   Actions

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              zjwang Zhijiang

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 4h 20m
                4h 20m

                  Issue deployment