Major Description
A broker stop or failure while deleting journal data files or archiving data files can result in partial deletion and no index updates.
Leading to errors on restart of the form:
java.io.IOException: Detected missing journal files.
ERROR BrokerService - Failed to start Apache ActiveMQ (localhost, null)
java.io.IOException: Detected missing journal files. [1, 2, 3, 4]
at org.apache.activemq.store.kahadb.MessageDatabase.recoverIndex(MessageDatabase.java:978)
at org.apache.activemq.store.kahadb.MessageDatabase$5.execute(MessageDatabase.java:714)
at org.apache.activemq.store.kahadb.disk.page.Transaction.execute(Transaction.java:779)
at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:711)
at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:464)
at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:483)
at org.apache.activemq.store.kahadb.MessageDatabase.doStart(MessageDatabase.java:297)
at org.apache.activemq.store.kahadb.KahaDBStore.doStart(KahaDBStore.java:218)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.doStart(KahaDBPersistenceAdapter.java:226)
at org.apache.activemq.util.ServiceSupport.start(ServiceSupport.java:55)
at org.apache.activemq.broker.BrokerService.doStartPersistenceAdapter(BrokerService.java:674)
at org.apache.activemq.broker.BrokerService.startPersistenceAdapter(BrokerService.java:658)
at org.apache.activemq.broker.BrokerService.start(BrokerService.java:622)
The file deletion of the gc candidates is part of the index transaction, so partial deletion can occur but the index does not get updated.
For gc, the index updates need to complete to remove references to the data files first. Only after a successful update to the index should the removal proceed. If the removal fails, on restart the orphaned data files will get gc ed.