Uploaded image for project: 'ActiveMQ'
  1. ActiveMQ
  2. AMQ-5173

Kaha DB cleanup fails to reclaim disk space after some time

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 5.6.0
    • Fix Version/s: 5.9.1
    • Component/s: None
    • Labels:
      None

      Description

      We are seeing an issue where a broker using Kaha DB will gradually increase its footprint on disk over time and never shrink back down again even when all queues are empty.

      The default cleanup settings will reclaim space at first but over time it seems the size of the data directory will grow and never shrink back down to its original size.

      I can reproduce this with a test that sends and consumes a large number of messages to a single queue (.net client, broker hosted on windows). The data file grows and will never shrink back down again.

              [Test]
              public void ButLoadOfMessagesOnASingleQueue()
              {
                  const int messagesPerProducerThread = 1000000;
                  const int numberProducers = 1;
                  const int numberConsumers = 3;
                  var producersConsumers = new List<Task>();
      
                  var factory = new ConnectionFactory
                  {
                      AcknowledgementMode = AcknowledgementMode.Transactional,
                      AsyncSend = true
                  };
      
                  for (var i = 0; i < numberProducers; i++)
                  {
                      var producer = Task.Factory.StartNew(() => Send(factory, messagesPerProducerThread));
                      producersConsumers.Add(producer);
                  }
                  for (var i = 0; i < numberConsumers; i++)
                  {
                      var consumer = Task.Factory.StartNew(() => Consume(factory));
                      producersConsumers.Add(consumer);
                  }
      
                  Task.WaitAll(producersConsumers.ToArray());
              }
      
              private void Send(IConnectionFactory connectionFactory, int noMessages)
              {
                  var connection = connectionFactory.CreateConnection();
                  connection.Start();
      
                  var session = connection.CreateSession();
                  var destination = SessionUtil.GetDestination(session, GetQueueName(1));
                  var producer = session.CreateProducer(destination);
                  for (var i = 0; i < noMessages; i++)
                  {
                      producer.Send(new ActiveMQTextMessage(i.ToString()));
      
                      if (i%100 == 0)
                          session.Commit();
                  }
                  session.Commit();
                  connection.Close();
              }
      
              private void Consume(IConnectionFactory connectionFactory)
              {
                  var connection = connectionFactory.CreateConnection();
                  connection.Start();
      
                  var session = connection.CreateSession();
                  var destination = SessionUtil.GetDestination(session, GetQueueName(1));
                  var consumer = session.CreateConsumer(destination);
                  var count = 0;
                  while (true)
                  {
                      if (consumer.Receive(TimeSpan.FromSeconds(5)) == null)
                          break;
                      count++;
      
                      if (count%100 == 0)
                          session.Commit();
                  }
                  session.Commit();
                  connection.Close();
              }
      

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              imran99 Imran
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: