Uploaded image for project: 'ActiveMQ Classic'
  1. ActiveMQ Classic
  2. AMQ-9343

Reduce inflight transaction memory footprint in KahaDB

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 6.0.0, 5.18.3, 5.17.6
    • KahaDB
    • None

    Description

      When transacted messages are sent KahaDB tracks the inflight and prepared (for XA) add commands in a map so that when the transaction is eventually committed the commands can be used to update the KahaDB index. Currently the entire message is kept inside the map while the transaction is pending. This is not a problem for small/quick transactions but for transactions that are long running and/or have a lot of pending (or big) messages before commit this can use up a decent amount of memory and cause GC pressure.

      To reduce memory we can go ahead and just clear the message from the command after it is stored in the journal. The pending operations that are queued up to be processed on commit only need metadata from the add command (such as message id and destination) so the actual messages are no longer needed and can be cleared. This is a pretty easy change and should help for edge cases with a large number of pending message sends or big messages.

      Note that on broker restart/recovery the messages are loaded again for pending transactions and processed (such as XA in the prepared state) and the messages are needed for processing recovery for XA transactions so the memory will be higher again as the inflight and prepared maps will track the entire message again (just like before this change) but this should be fine as it is just during recovery and once finished and the broker is online again it will clear the new incoming messages.

      Attachments

        Activity

          People

            cshannon Christopher L. Shannon
            cshannon Christopher L. Shannon
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1h
                1h