Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-9876

Implement Raft Protocol for Metadata Quorum

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core
    • Labels:
      None

      Description

      This tracks the completion of the Raft Protocol specified in KIP-595: https://cwiki.apache.org/confluence/display/KAFKA/KIP-595%3A+A+Raft+Protocol+for+the+Metadata+Quorum. If/when the KIP is approved by the community, we will create smaller sub-tasks to track overall prgress.

        Attachments

          Issue Links

          1.
          Fetch protocol changes for KIP-595 Sub-task Resolved Jason Gustafson
          2.
          Fix edge case in Raft truncation protocol Sub-task Resolved Jason Gustafson
          3.
          Core Raft implementation Sub-task Resolved Jason Gustafson
          4.
          Fix minor behavior difference in `MockLog` Sub-task Resolved Jason Gustafson
          5.
          Unit tests for VotedState Sub-task Resolved Jason Gustafson
          6.
          Explore performance impact of leader fsync deferral Sub-task Open Sagar Rao
          7.
          Voters should always initialize as followers Sub-task Resolved Jason Gustafson
          8.
          Add log flush semantics to simulation test Sub-task Resolved Jason Gustafson
          9.
          Enhance Raft exception handling Sub-task Open HaiyuanZhao
          10.
          Add linger semantics to raft Sub-task Resolved Jason Gustafson
          11.
          Raft client should push all committed data to listeners Sub-task Resolved Jason Gustafson
          12.
          LeaderChangeMessage should include the leader as one of the voters Sub-task Resolved Sagar Rao
          13.
          Bypass log validation for writes to raft log Sub-task Resolved Haoran Xuan
          14.
          Add resigned state to raft state machine to preserve leader/epoch information Sub-task Resolved Jason Gustafson
          15.
          Fix VotedToUnattached test error Sub-task Resolved dengziming
          16.
          Raft leader should flush accumulated writes after a min size is reached Sub-task Open Sagar Rao
          17.
          Raft leader should resign after write failures Sub-task Open Boyang Chen
          18.
          Decide whether Raft listener callback errors are fatal Sub-task Open Unassigned
          19.
          Complete fetches in purgatory immediately after raft leader resigns Sub-task Resolved dengziming
          20.
          Add missing unit test for `UnattachedState` Sub-task Resolved dengziming
          21.
          Add clusterId validation to Fetch handling Sub-task Resolved David Jacot
          22.
          Ensure raft io thread wakes up after linger expiration Sub-task Resolved Jason Gustafson
          23.
          Consider using "acknowledged" over "endorsing" for voters which have recognized the current leader Sub-task Resolved Sagar Rao
          24.
          Refactor raft outbound request channel Sub-task Resolved Jason Gustafson
          25.
          Consider better return type of RaftClient.scheduleAppend Sub-task Resolved dengziming
          26.
          Raft observers should not require an id to fetch Sub-task Resolved Jason Gustafson
          27.
          Loosen monotonic fetch offset validation by raft leader Sub-task Resolved Jason Gustafson
          28.
          Support non-routable quorum voter addresses Sub-task Resolved Alok Nikhil
          29.
          Change the BatchAccumulator to split records into batches Sub-task Resolved Jose Armando Garcia Sancio
          30.
          KafkaRaftClient should use the LEO when appending LeaderChangeMessage Sub-task Resolved Jose Armando Garcia Sancio
          31.
          KafkaMetadataLog and MockLock should validate that appended epochs are monotonically Sub-task Open HaiyuanZhao
          32.
          Improve KafkaRaftClient handling of graceful shutdown Sub-task Open Sagar Rao
          33.
          ClusterId validation for Vote, BeginQourum and EndQuorum Sub-task Resolved dengziming
          34.
          Improve raft simulation tests Sub-task Resolved Jason Gustafson
          35.
          Raft should prevent truncation below high watermark Sub-task Resolved Jason Gustafson
          36.
          Extend LogManager to cover the metadata topic Sub-task Open HaiyuanZhao
          37.
          Decide whether inconsistent cluster id error are fatal Sub-task Open dengziming
          38.
          Allow votes to be granted in resigned state Sub-task Resolved dengziming
          39.
          Support api to resign raft leadership Sub-task Resolved Jason Gustafson
          40.
          Use UncheckedIOException when wrapping IOException Sub-task Resolved loboxu
          41.
          Add integration tests for "test-kraft-server-start" Sub-task Open HaiyuanZhao
          42.
          Notify listeners of raft client shutdowns Sub-task Open HaiyuanZhao
          43.
          Make kraft configuration properties public Sub-task Resolved HaiyuanZhao
          44.
          Remove the method RaftClient.leaderAndEpoch Sub-task Open HaiyuanZhao
          45.
          Revisit the usage of BufferSuppliers in Kraft Sub-task Open Jose Armando Garcia Sancio
          46.
          Validate node id, process role and quorum voters Sub-task Resolved Ryan Dielhenn

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: