[AMQ-4743] LevelDB client crashes when committing transaction

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Affects Version/s: 5.9.0
• Component/s: LevelDB

Environment

• Windows Server 2008 R2
• java 1.6.0_38 64bit
• levelDB storage with default settings

• Assignee: Unassigned
• Reporter: Anton Gnutov
• Created: 27/Sep/13 08:38
• Updated: 31/Jan/17 19:43
• Resolved: 31/Jan/17 19:43

Description

Server craches during work. It worked correctly for a few days before.

2013-09-27 04:43:09,138 | INFO  | Stopping BrokerService[AGS] due to exception, java.nio.channels.ClosedChannelException | org.apache.activemq.util.DefaultIOExceptionHandler | LevelDB IOException handler.
java.nio.channels.ClosedChannelException
	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:629)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply$mcV$sp(RecordLog.scala:207)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply(RecordLog.scala:202)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply(RecordLog.scala:202)
	at org.apache.activemq.leveldb.util.TimeMetric.apply(TimeMetric.scala:43)
	at org.apache.activemq.leveldb.RecordLog$LogAppender.flush(RecordLog.scala:202)
	at org.apache.activemq.leveldb.RecordLog.appender(RecordLog.scala:476)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply$mcV$sp(LevelDBClient.scala:1100)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.LevelDBClient.usingIndex(LevelDBClient.scala:958)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$might_fail_using_index$1.apply(LevelDBClient.scala:964)
	at org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:533)
	at org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:964)
	at org.apache.activemq.leveldb.LevelDBClient.removeCollection(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.DBManager.removeTransactionContainer(DBManager.scala:790)
	at org.apache.activemq.leveldb.LevelDBStore.commit(LevelDBStore.scala:445)
	at org.apache.activemq.transaction.XATransaction.storeCommit(XATransaction.java:85)
	at org.apache.activemq.transaction.XATransaction.commit(XATransaction.java:75)
	at org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:263)
	at org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:112)
	at org.apache.activemq.broker.TransportConnection.processCommitTransactionTwoPhase(TransportConnection.java:435)
	at org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:102)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:149)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
	at java.lang.Thread.run(Thread.java:662)
2013-09-27 04:43:09,153 | INFO  | Apache ActiveMQ 5.9-SNAPSHOT (AGS, ID:spb01-i01-dsh01-59671-1380027924305-0:1) is shutting down | org.apache.activemq.broker.BrokerService | IOExceptionHandler: stopping BrokerService[AGS]
2013-09-27 04:43:09,497 | INFO  | Stopped LevelDB[C:\apache-activemq-5.9-SNAPSHOT\bin\win64\..\..\data\leveldb] | org.apache.activemq.leveldb.LevelDBStore | ActiveMQ Transport: tcp:///192.168.33.252:50929@61616
2013-09-27 04:43:09,497 | WARN  | Store COMMIT FAILED:  | org.apache.activemq.transaction.XATransaction | ActiveMQ Transport: tcp:///192.168.33.252:50929@61616
java.nio.channels.ClosedChannelException
	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
	at sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:629)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply$mcV$sp(RecordLog.scala:207)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply(RecordLog.scala:202)
	at org.apache.activemq.leveldb.RecordLog$LogAppender$$anonfun$flush$1.apply(RecordLog.scala:202)
	at org.apache.activemq.leveldb.util.TimeMetric.apply(TimeMetric.scala:43)
	at org.apache.activemq.leveldb.RecordLog$LogAppender.flush(RecordLog.scala:202)
	at org.apache.activemq.leveldb.RecordLog.appender(RecordLog.scala:476)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply$mcV$sp(LevelDBClient.scala:1100)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$removeCollection$1.apply(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.LevelDBClient.usingIndex(LevelDBClient.scala:958)
	at org.apache.activemq.leveldb.LevelDBClient$$anonfun$might_fail_using_index$1.apply(LevelDBClient.scala:964)
	at org.apache.activemq.leveldb.LevelDBClient.might_fail(LevelDBClient.scala:533)
	at org.apache.activemq.leveldb.LevelDBClient.might_fail_using_index(LevelDBClient.scala:964)
	at org.apache.activemq.leveldb.LevelDBClient.removeCollection(LevelDBClient.scala:1099)
	at org.apache.activemq.leveldb.DBManager.removeTransactionContainer(DBManager.scala:790)
	at org.apache.activemq.leveldb.LevelDBStore.commit(LevelDBStore.scala:445)
	at org.apache.activemq.transaction.XATransaction.storeCommit(XATransaction.java:85)
	at org.apache.activemq.transaction.XATransaction.commit(XATransaction.java:75)
	at org.apache.activemq.broker.TransactionBroker.commitTransaction(TransactionBroker.java:263)
	at org.apache.activemq.broker.MutableBrokerFilter.commitTransaction(MutableBrokerFilter.java:112)
	at org.apache.activemq.broker.TransportConnection.processCommitTransactionTwoPhase(TransportConnection.java:435)
	at org.apache.activemq.command.TransactionInfo.visit(TransactionInfo.java:102)
	at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:294)
	at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:149)
	at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
	at org.apache.activemq.transport.WireFormatNegotiator.onCommand(WireFormatNegotiator.java:113)
	at org.apache.activemq.transport.AbstractInactivityMonitor.onCommand(AbstractInactivityMonitor.java:288)
	at org.apache.activemq.transport.TransportSupport.doConsume(TransportSupport.java:83)
	at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:214)
	at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:196)
	at java.lang.Thread.run(Thread.java:662)

Issue Links

• is related to: AMQ-4882 LevelDB can get to a corrupt state when using XA transactions