Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10310

Kafka Raft Snapshot

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: In Progress
    • Major
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      Tracking issue for KIP-630: Kafka Raft Snapshot

      Attachments

        Issue Links

          blocks

          New Feature - A new feature of the product, which has yet to be developed. KAFKA-12466 Controller and Broker Metadata Snapshots

          • Major - Major loss of function.
          • In Progress
          mentioned in

          Page Loading...

          1.
          		 Update start offset and end offset of the replicated log Sub-task Resolved Jose Armando Garcia Sancio
          2.
          		 Message for fetch snapshot and fetch Sub-task Resolved Jose Armando Garcia Sancio
          3.
          		 Internal API for generating snapshots Sub-task Resolved Jose Armando Garcia Sancio
          4.
          		 Implement FetchSnapshot RPC Sub-task Resolved Jose Armando Garcia Sancio
          5.
          		 In Raft support nonzero start offset and SnapshotId for the Fetch request handling Sub-task Resolved Jose Armando Garcia Sancio
          6.
          		 API and implementation for snapshot loading Sub-task Resolved Jose Armando Garcia Sancio
          7.
          		 Validate the snapshot id when the state machine creates a snapshot Sub-task Resolved Haoran Xuan
          8.
          		 Delay increasing the log start offset Sub-task Resolved David Arthur
          9.
          		 Add toString methods to some of the classes introduced by this Epic Sub-task Open Unassigned
          10.
          		 Implement zero copy for FetchSnapshot Sub-task Resolved Deng Ziming
          11.
          		 The freeze operation should validate the content of the snapshot Sub-task Open HaiyuanZhao
          12.
          		 Send cluster id information with the FetchSnapshot request Sub-task Resolved Rohit Deshpande
          13.
          		 Limit the size of Fetch and FetchSnapshot response based on broker configuration Sub-task Open HaiyuanZhao
          14.
          		 Add metrics enumerated in KIP-630 Sub-task Open loboxu
          15.
          		 Delete snapshots if it's end offset is less than the log start offset Sub-task Resolved Deng Ziming
          16.
          		 Add tests that cover all of all of the cases for ReplicatedLog::validateOffsetAndEpoch Sub-task Resolved Rohit Deshpande
          17.
          		 Validate the snapshot id when the state machine freeze a snapshot Sub-task Open loboxu
          18.
          		 Investigate allowing for Fetch request that are not validated Sub-task Open Unassigned
          19.
          		 Re-design the ownership model for snapshots Sub-task Resolved Jose Armando Garcia Sancio
          20.
          		 Move the BatchAccumulator in KafkaRaftClient to LeaderState Sub-task Resolved Ryan Dielhenn
          21.
          		 Log truncation due to divergence should also remove snapshots Sub-task Open HaiyuanZhao
          22.
          		 Load snapshot heuristic Sub-task Open Deng Ziming
          23.
          		 Interfaces for SnapshotReader and SnapshotWriter Sub-task Resolved loboxu
          24.
          		 Add simulation invariant for leadership and snapshot Sub-task Resolved HaiyuanZhao
          25.
          		 Metadata Snapshot File Delimiters Sub-task Resolved Niket Goel
          26.
          		 Update KIP and dev mailing list Sub-task Resolved Jose Armando Garcia Sancio
          27.
          		 Expose log record append time to the controller/broker Sub-task Resolved Jose Armando Garcia Sancio
          28.
          		 Interpret snapshot header version to correctly parse the snapshot Sub-task Open loboxu
          29.
          		 SnapshotReader should decode and report the append time in the header Sub-task Resolved Jose Armando Garcia Sancio
          30.
          		 Kafka-dump-log needs to support snapshot records Sub-task In Progress Jose Armando Garcia Sancio
          31.
          		 Support different compression type for snapshots Sub-task Open Jose Armando Garcia Sancio
          32.
          		 Check CRC when reading snapshots Sub-task Resolved Jose Armando Garcia Sancio
          33.
          		 Add support for reading KRaft snapshots in kafka-dump-log Sub-task Resolved Deng Ziming
          34.
          		 Implement the snapshot cleanup policy Sub-task Open Jose Armando Garcia Sancio
          35.
          		 Heuristic for increasing the log start offset after replicas are caught up Sub-task Open José Armando García Sancio
          36.
          		 KRaft snapshot freeze should never perform copy Sub-task Open José Armando García Sancio

          Activity

            People

              jagsancio Jose Armando Garcia Sancio
              jagsancio Jose Armando Garcia Sancio
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: