Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Abandoned
-
None
-
None
-
None
Description
- Entry point for transaction requests
- Appends transaction control records to the transaction journal
- Sends transaction control records to data brokers
- Responsible for expiring transactions
- Supports fail-over: for which it needs to maintain a transaction HW which is the offset of the BEGIN control record of the earliest pending transaction. It should checkpoint the HW periodically either to ZK/separate topic/offset commit.
We merge KAFKA-1565 transaction manager failover handling into this JIRA. Transaction manager should guarantee that, once a pre-commit/pre-abort request is acknowledged, commit/abort request will be delivered to partitions involved in the transaction.
This patch handles the following failover scenarios:
1) Transaction manager or its followers fail before txRequest is duplicated on local log and followers.
Solution: Transaction manager responds to request with error status. The producer keeps trying to commit.
2) The txPartition’s leader is not available.
Solution: Put txRequest on unSentTxRequestQueue. When metadataCache is updated, check and re-send txRequest from unSentTxRequestQueue if possible.
3) The txPartition’s leader fails when txRequest is in channel manager.
Solution: Retrieve all txRequests queued for transmission to this broker and put them on unSentTxRequestQueue.
4) Transaction manage does not receive success response from txPartition’s leaders within timeout period.
Solution: Transaction manager expires the txRequest and re-send it.
5) Transaction manager fails.
Solution: The new transaction manager reads transactionHW from zookeeper, and sends txRequest starting from the transactionHW.
This patch does not provide the following feature. These will be provided in separate patches.
1) Producer offset commit.
2) Transaction expiration.