Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-5059 Implement Transactional Coordinator
  3. KAFKA-5151

Refactor TransactionCoordinator in-memory structure and error handling logic

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 0.11.0.0
    • core
    • None

    Description

      Current status:

      1. we are having two types of threads: request handling thread for any client requests as well as controller requests for `immigration` and `emigration`, and the marker sender thread for draining queued marker entries and handle responses. They maintain different in-memory cache structures like the `txnMetadataCache`, and the `pendingTxnMap` which are storing the same info, and they access some of the shared structures concurrently, like the markers queue and the markerPurgatory.

      2. we are having one queue per broker today, and due to the emigration purpose we probably are having one queue per brokerId + TxnLogPartitionId + DataPartitionId, which would result in a lot of queues to handle.

      This ticket is for collapsing some of these structures and simplify the access of them from concurrent threads.

      Attachments

        Activity

          People

            guozhang Guozhang Wang
            guozhang Guozhang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: