[TAJO-1279] Cleanup TajoAsyncDispatcher and interrupt stop events

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.8.0, 0.9.0
• Fix Version/s: 0.10.0
• Component/s: QueryMaster
• Assignee: Jinho Kim
• Reporter: Jinho Kim
• Created: 06/Jan/15 03:22
• Updated: 27/Mar/15 07:04
• Resolved: 08/Jan/15 07:21

Description

See the titile.
TajoAsyncDispatcher is an implementation copied from yarn AsyncDispatcher for log message handling. but too many create the thread interrupted message.
We should improve the stop event.

2014-12-17 10:08:35,327 WARN: org.apache.tajo.master.TajoAsyncDispatcher (stop(115)) - Interrupted Exception while stopping
2014-12-17 10:08:35,328 WARN: org.apache.tajo.master.TajoAsyncDispatcher (stop(115)) - Interrupted Exception while stopping
2014-12-17 10:08:36,896 WARN: org.apache.tajo.master.TajoAsyncDispatcher (stop(115)) - Interrupted Exception while stopping
2014-12-17 10:08:36,898 WARN: org.apache.tajo.master.TajoAsyncDispatcher (stop(115)) - Interrupted Exception while stopping
2014-12-17 10:08:37,745 FATAL: org.apache.tajo.master.TajoAsyncDispatcher (dispatch(143)) - Error in dispatcher thread:QUERY_JOB_HEARTBEAT
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.InterruptedException
	at org.apache.tajo.master.TajoAsyncDispatcher$GenericEventHandler.handle(TajoAsyncDispatcher.java:204)
	at org.apache.tajo.master.querymaster.QueryInProgress.heartbeat(QueryInProgress.java:290)
	at org.apache.tajo.master.querymaster.QueryInProgress.access$000(QueryInProgress.java:53)
	at org.apache.tajo.master.querymaster.QueryInProgress$QueryInProgressEventHandler.handle(QueryInProgress.java:196)
	at org.apache.tajo.master.querymaster.QueryInProgress$QueryInProgressEventHandler.handle(QueryInProgress.java:192)
	at org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
	at org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
	at java.lang.Thread.run(Thread.java:701)
Caused by: java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1222)
	at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
	at java.util.concurrent.LinkedBlockingQueue.put(LinkedBlockingQueue.java:269)
	at org.apache.tajo.master.TajoAsyncDispatcher$GenericEventHandler.handle(TajoAsyncDispatcher.java:199)
	... 7 more
2014-12-17 10:08:37,746 WARN: org.apache.tajo.master.TajoAsyncDispatcher (stop(115)) - Interrupted Exception while stopping

Issue Links

• contains: TAJO-1138 TestKillQuery.testKillQueryFromInitState fails occasionally.
• is related to: TAJO-1251 Query is hanging occasionally by shuffle report