[AMQ-5173] Kaha DB cleanup fails to reclaim disk space after some time

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.6.0
• Fix Version/s: 5.9.1
• Assignee: Unassigned
• Reporter: Imran
• Created: 06/May/14 14:42
• Updated: 07/May/14 09:10
• Resolved: 07/May/14 09:10

Description

We are seeing an issue where a broker using Kaha DB will gradually increase its footprint on disk over time and never shrink back down again even when all queues are empty.

The default cleanup settings will reclaim space at first but over time it seems the size of the data directory will grow and never shrink back down to its original size.

I can reproduce this with a test that sends and consumes a large number of messages to a single queue (.net client, broker hosted on windows). The data file grows and will never shrink back down again.

        [Test]
        public void ButLoadOfMessagesOnASingleQueue()
        {
            const int messagesPerProducerThread = 1000000;
            const int numberProducers = 1;
            const int numberConsumers = 3;
            var producersConsumers = new List<Task>();

            var factory = new ConnectionFactory
            {
                AcknowledgementMode = AcknowledgementMode.Transactional,
                AsyncSend = true
            };

            for (var i = 0; i < numberProducers; i++)
            {
                var producer = Task.Factory.StartNew(() => Send(factory, messagesPerProducerThread));
                producersConsumers.Add(producer);
            }
            for (var i = 0; i < numberConsumers; i++)
            {
                var consumer = Task.Factory.StartNew(() => Consume(factory));
                producersConsumers.Add(consumer);
            }

            Task.WaitAll(producersConsumers.ToArray());
        }

        private void Send(IConnectionFactory connectionFactory, int noMessages)
        {
            var connection = connectionFactory.CreateConnection();
            connection.Start();

            var session = connection.CreateSession();
            var destination = SessionUtil.GetDestination(session, GetQueueName(1));
            var producer = session.CreateProducer(destination);
            for (var i = 0; i < noMessages; i++)
            {
                producer.Send(new ActiveMQTextMessage(i.ToString()));

                if (i%100 == 0)
                    session.Commit();
            }
            session.Commit();
            connection.Close();
        }

        private void Consume(IConnectionFactory connectionFactory)
        {
            var connection = connectionFactory.CreateConnection();
            connection.Start();

            var session = connection.CreateSession();
            var destination = SessionUtil.GetDestination(session, GetQueueName(1));
            var consumer = session.CreateConsumer(destination);
            var count = 0;
            while (true)
            {
                if (consumer.Receive(TimeSpan.FromSeconds(5)) == null)
                    break;
                count++;

                if (count%100 == 0)
                    session.Commit();
            }
            session.Commit();
            connection.Close();
        }