[ASTERIXDB-1425] Sporadic test hang (LSN is out of bounds)

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Component/s: TX - Transactions
• Assignee: Murtadha Makki Al Hubail
• Reporter: Yingyi Bu
• Created: 06/May/16 01:19
• Updated: 20/May/16 23:21
• Resolved: 20/May/16 23:21

Description

I run into this sporadic test hanging:

org.apache.asterix.transaction.management.service.transaction.TransactionManager abortTransaction
SEVERE: Could not complete rollback! System is in an inconsistent state
org.apache.asterix.common.exceptions.ACIDException: LSN is out of bounds
	at org.apache.asterix.transaction.management.service.logging.LogReader.read(LogReader.java:241)
	at org.apache.asterix.transaction.management.service.recovery.RecoveryManager.rollbackTransaction(RecoveryManager.java:796)
	at org.apache.asterix.transaction.management.service.transaction.TransactionManager.abortTransaction(TransactionManager.java:64)
	at org.apache.asterix.transaction.management.service.transaction.TransactionManager.completedTransaction(TransactionManager.java:130)
	at org.apache.asterix.runtime.job.listener.JobEventListenerFactory$1.jobletFinish(JobEventListenerFactory.java:58)
	at org.apache.hyracks.control.nc.Joblet.performCleanup(Joblet.java:317)
	at org.apache.hyracks.control.nc.Joblet.removeTask(Joblet.java:153)
	at org.apache.hyracks.control.nc.work.NotifyTaskFailureWork.run(NotifyTaskFailureWork.java:54)
	at org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:132)
Exception in thread "Thread-42" java.lang.Error: org.apache.asterix.common.exceptions.ACIDException: Could not complete rollback! System is in an inconsistent state
	at org.apache.asterix.runtime.job.listener.JobEventListenerFactory$1.jobletFinish(JobEventListenerFactory.java:61)
	at org.apache.hyracks.control.nc.Joblet.performCleanup(Joblet.java:317)
	at org.apache.hyracks.control.nc.Joblet.removeTask(Joblet.java:153)
	at org.apache.hyracks.control.nc.work.NotifyTaskFailureWork.run(NotifyTaskFailureWork.java:54)
	at org.apache.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:132)
Caused by: org.apache.asterix.common.exceptions.ACIDException: Could not complete rollback! System is in an inconsistent state
	at org.apache.asterix.transaction.management.service.transaction.TransactionManager.abortTransaction(TransactionManager.java:72)
	at org.apache.asterix.transaction.management.service.transaction.TransactionManager.completedTransaction(TransactionManager.java:130)
	at org.apache.asterix.runtime.job.listener.JobEventListenerFactory$1.jobletFinish(JobEventListenerFactory.java:58)
	... 4 more
Caused by: org.apache.asterix.common.exceptions.ACIDException: LSN is out of bounds
	at org.apache.asterix.transaction.management.service.logging.LogReader.read(LogReader.java:241)
	at org.apache.asterix.transaction.management.service.recovery.RecoveryManager.rollbackTransaction(RecoveryManager.java:796)
	at org.apache.asterix.transaction.management.service.transaction.TransactionManager.abortTransaction(TransactionManager.java:64)
	... 6 more