Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-3260

Reduce the on-disk footprint of transaction metadata in participants

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: server, transactions
    • Labels:
      None

      Description

      We should remove the commit timestamp and txn metadata once we've flushed all in-memory stores that rely on metadata for determining commit timestamp. If we see tablets serving more frequent transactions, the persisted metadata may become less negligible.

      One thing to watch out for here is that we currently use metadata to determine whether a transaction has ever existed on the participant – if we simply get rid of the metadata, we will lose out on this knowledge. More thought should be given to how and when to do this safely, and ensure that we clean up the metadata in such a way that no invariants are broken with regards to knowing about transaction existence (e.g. perhaps only clean up the txn metadata if the corresponding TxnStatusManager has been deleted?).

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              awong Andrew Wong
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: