Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10310

Kafka Raft Snapshot

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Tracking issue for KIP-630: Kafka Raft Snapshot

      Attachments

        Issue Links

        blocks

        New Feature - A new feature of the product, which has yet to be developed. KAFKA-12466 Controller and Broker Metadata Snapshots

        • Major - Major loss of function.
        • In Progress
        mentioned in

        Page Loading...

        1.
        		 Update start offset and end offset of the replicated log Sub-task Resolved Jose Armando Garcia Sancio Actions
        2.
        		 Message for fetch snapshot and fetch Sub-task Resolved Jose Armando Garcia Sancio Actions
        3.
        		 Internal API for generating snapshots Sub-task Resolved Jose Armando Garcia Sancio Actions
        4.
        		 Implement FetchSnapshot RPC Sub-task Resolved Jose Armando Garcia Sancio Actions
        5.
        		 In Raft support nonzero start offset and SnapshotId for the Fetch request handling Sub-task Resolved Jose Armando Garcia Sancio Actions
        6.
        		 API and implementation for snapshot loading Sub-task Resolved Jose Armando Garcia Sancio Actions
        7.
        		 Validate the snapshot id when the state machine creates a snapshot Sub-task Resolved Haoran Xuan Actions
        8.
        		 Delay increasing the log start offset Sub-task Resolved David Arthur Actions
        9.
        		 Add toString methods to some of the classes introduced by this Epic Sub-task Open Unassigned Actions
        10.
        		 Implement zero copy for FetchSnapshot Sub-task Resolved Deng Ziming Actions
        11.
        		 The freeze operation should validate the content of the snapshot Sub-task Open HaiyuanZhao Actions
        12.
        		 Send cluster id information with the FetchSnapshot request Sub-task Resolved Rohit Deshpande Actions
        13.
        		 Limit the size of Fetch and FetchSnapshot response based on broker configuration Sub-task Open HaiyuanZhao Actions
        14.
        		 Add metrics enumerated in KIP-630 Sub-task Open loboxu Actions
        15.
        		 Delete snapshots if it's end offset is less than the log start offset Sub-task Resolved Deng Ziming Actions
        16.
        		 Add tests that cover all of all of the cases for ReplicatedLog::validateOffsetAndEpoch Sub-task Resolved Rohit Deshpande Actions
        17.
        		 Validate the snapshot id when the state machine freeze a snapshot Sub-task Open loboxu Actions
        18.
        		 Investigate allowing for Fetch request that are not validated Sub-task Open Unassigned Actions
        19.
        		 Re-design the ownership model for snapshots Sub-task Resolved Jose Armando Garcia Sancio Actions
        20.
        		 Move the BatchAccumulator in KafkaRaftClient to LeaderState Sub-task Resolved Ryan Dielhenn Actions
        21.
        		 Log truncation due to divergence should also remove snapshots Sub-task Open HaiyuanZhao Actions
        22.
        		 Load snapshot heuristic Sub-task Open Deng Ziming Actions
        23.
        		 Interfaces for SnapshotReader and SnapshotWriter Sub-task Resolved loboxu Actions
        24.
        		 Add simulation invariant for leadership and snapshot Sub-task Resolved HaiyuanZhao Actions
        25.
        		 Metadata Snapshot File Delimiters Sub-task Resolved Niket Goel Actions
        26.
        		 Update KIP and dev mailing list Sub-task Resolved Jose Armando Garcia Sancio Actions
        27.
        		 Expose log record append time to the controller/broker Sub-task Resolved Jose Armando Garcia Sancio Actions
        28.
        		 Interpret snapshot header version to correctly parse the snapshot Sub-task Open loboxu Actions
        29.
        		 SnapshotReader should decode and report the append time in the header Sub-task Resolved Jose Armando Garcia Sancio Actions
        30.
        		 Kafka-dump-log needs to support snapshot records Sub-task In Progress Jose Armando Garcia Sancio Actions
        31.
        		 Support different compression type for snapshots Sub-task Open Jose Armando Garcia Sancio Actions
        32.
        		 Check CRC when reading snapshots Sub-task Resolved Jose Armando Garcia Sancio Actions
        33.
        		 Add support for reading KRaft snapshots in kafka-dump-log Sub-task Resolved Deng Ziming Actions
        34.
        		 Implement the snapshot cleanup policy Sub-task Open Jose Armando Garcia Sancio Actions
        35.
        		 Heuristic for increasing the log start offset after replicas are caught up Sub-task Open José Armando García Sancio Actions
        36.
        		 KRaft snapshot freeze should never perform copy Sub-task Open José Armando García Sancio Actions

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jagsancio Jose Armando Garcia Sancio
            jagsancio Jose Armando Garcia Sancio
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:

              Slack

                Issue deployment