Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-4262

Pending send transaction can block on store usage and not resume once backlog is cleared

    XMLWordPrintableJSON

    Details

      Description

      A transacted send can block pending store usage, an active transaction will be visible to the store. This will block kahadb from gc of data files that occur after the inflight transaction.
      Consumers acking the pending messages can create additional data files that can't be gced. This is a problem because the queue can drain and not release the producer.
      With message database trace is is possible to see:

      14:20:20,515 | INFO  | .0.1:51273@61616 | Queue                            | 125 - org.apache.activemq.activemq-core - 5.7.0.fuse-71-047 | Usage(default:store:queue://flood_out:store) percentUsage=99%, usage=30447772, limit=20971520, percentUsageMinDelta=1%;Parent:Usage(default:store) percentUsage=145%, usage=30447772, limit=20971520, percentUsageMinDelta=1%: Persistent store is Full, 100% of 20971520. Stopping producer (ID:..) to prevent flooding queue://flood_out. See http://activemq.apache.org/producer-flow-control.html for more info (blocking for: 5650s)
      .....
      
      14:20:45,016 | TRACE | heckpoint Worker | MessageDatabase                  | 125 - org.apache.activemq.activemq-core - 5.7.0.fuse-71-047 | Last update: 30:453228, full gc candidates set: [11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30]
      14:20:45,016 | TRACE | heckpoint Worker | MessageDatabase                  | 125 - org.apache.activemq.activemq-core - 5.7.0.fuse-71-047 | gc candidates after first tx:11:860775, []
      14:20:45,016 | TRACE | heckpoint Worker | MessageDatabase                  | 125 - org.apache.activemq.activemq-core - 5.7.0.fuse-71-047 | gc candidates: []
      

      Here there is a build up of data files, after the inflight transaction, containing acks and messages that could be cleaned up.

      The inflight transactions should protect a range of data files that they reference, not all subsequent data files.

        Attachments

          Activity

            People

            • Assignee:
              gtully Gary Tully
              Reporter:
              gtully Gary Tully
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: