Details
-
Sub-task
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
None
Description
While implementing the EOS design, there are some minor (or major?) tweaks we made as we hands-on the code bases. We will compile all these changes in a single run at the end of the code-ready and this JIRA is for keeping track of all the changes we made.
02/27/2017: collapse the two types of transactional log messages into a single type with key as transactional id, and value as [pid, epoch, transaction_timeout, transaction_state, [topic partition ] ]. Also using a single memory map in cache instead of two on the TC.
03/01/2017: for pid expiration, we decided to use min(transactional id expiration timeout, topic retention). For topics enabled for compaction only, we just use the transactional timeout. If the retention setting is larger than the transactional id expiration timeout, then the pid will be "logically" expired (i.e. we will remove it from the cached pid mapping and ignore it when rebuilding the cache from the log)
03/20/2017: add a new exception type in `o.a.k.common.errors` for invalid transaction timeout values.
03/25/2017: extend WriteTxnMarkerRequest to contain multiple markers for multiple PIDs with a single request.
04/20/2017: add transactionStartTime to TransactionMetadata
04/26/2017: added a new retriable error: Errors.CONCURRENT_TRANSACTIONS
04/01/2017: We also enforce acks=all on the client when idempotence is enabled. Without this, we cannot again guarantee idemptoence.
04/10/2017: WE also don't pass the underlying exception to `RetriableOffsetCommitFailedException` anymore: https://issues.apache.org/jira/browse/KAFKA-5052