[AMQ-3122] Recovery after out of disk space (when space freed up) needs manual intervention

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.4.1, 5.4.2
• Fix Version/s: 5.5.0
• Component/s: Message Store
• Assignee: Gary Tully
• Reporter: Gary Tully
• Created: 07/Jan/11 14:08
• Updated: 21/Jan/11 17:50
• Resolved: 07/Jan/11 15:00

Description

When broker runs out of disk space, the journal excepts and the send fails with an exception as expected. The default IO exception handler ignores the exception. (aside: it is possible to install an exception handler that shuts down the broker: http://activemq.apache.org/configurable-ioexception-handling.html)

On restart, the last journal file that failed a write is left with 0 size, this impedes restart recovery as it cannot be parsed.

ERROR | Failed to start ActiveMQ JMS Message Broker. Reason: java.io.EOFException
java.io.EOFException
	at java.io.RandomAccessFile.readInt(RandomAccessFile.java:725)
	at org.apache.kahadb.journal.DataFileAccessor.readLocationDetails(DataFileAccessor.java:115)
	at org.apache.kahadb.journal.Journal.getNextLocation(Journal.java:498)
	at org.apache.activemq.store.kahadb.MessageDatabase.recover(MessageDatabase.java:395)
	at org.apache.activemq.store.kahadb.MessageDatabase.open(MessageDatabase.java:280)
	at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:321)
	at org.apache.activemq.store.kahadb.MessageDatabase.start(MessageDatabase.java:182)
	at org.apache.activemq.store.kahadb.KahaDBPersistenceAdapter.start(KahaDBPersistenceAdapter.java:183)
	at org.apache.activemq.broker.BrokerService.start(BrokerService.java:469)

Issue Links

• is required by: AMQ-3120 KahaDB error: "Could not locate data file"