[AMQ-6625] IOException in kahaDB needs to pause pending IOExceptionHandler intervention

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.14.0
• Fix Version/s: 5.15.0
• Component/s: KahaDB
• Assignee: Gary Tully
• Reporter: Gary Tully
• Created: 13/Mar/17 13:39
• Updated: 27/Jul/18 12:18
• Resolved: 15/Jun/17 16:36

Description

KahaDB can recover from kill -9 by replaying the journal from the last checkpoint or by detecting and reapplying partial writes to the index.
However this activity is compromised if the journal or index accepts subsequent writes. It an lead to skipped write batches or skipped partial updates to the index.
The desirable behaviour of treating an IOException as fatal and stopping the broker in the knowledge that it will restart and fully recover needs to treat the first IO error as fatal and by default not accept any further writes.
A more advanced IOException handler can facilitate staying alive in more specific scenarios and reactivate kahadb.

Issue Links

• causes: AMQ-7010 No space left on device leading to broker shutdown even if ignored
• is related to: AMQ-3725 Kahadb error during SAN failover delayed write - Allow kahaDB to recover in a similar manner as the JDBC store using the IOExceptionHandler
• is related to: AMQ-6606 Journal partial write can result in batch corruption on restart
• is related to: AMQ-2042 Shutdown broker if default message store cannot access the disk