[KUDU-3260] Reduce the on-disk footprint of transaction metadata in participants

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: server, transactions
• Assignee: Unassigned
• Reporter: Andrew Wong
• Created: 12/Mar/21 00:12
• Updated: 12/Mar/21 00:12

Description

We should remove the commit timestamp and txn metadata once we've flushed all in-memory stores that rely on metadata for determining commit timestamp. If we see tablets serving more frequent transactions, the persisted metadata may become less negligible.

One thing to watch out for here is that we currently use metadata to determine whether a transaction has ever existed on the participant – if we simply get rid of the metadata, we will lose out on this knowledge. More thought should be given to how and when to do this safely, and ensure that we clean up the metadata in such a way that no invariants are broken with regards to knowing about transaction existence (e.g. perhaps only clean up the txn metadata if the corresponding TxnStatusManager has been deleted?).