[TAJO-1726] Error in a running TaskAttempt may cause invalid event.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Fix Version/s: 0.11.0
• Component/s: QueryMaster
• Assignee: Unassigned
• Reporter: Hyunsik Choi
• Created: 30/Jul/15 08:36
• Updated: 04/Sep/15 01:56
• Resolved: 04/Sep/15 01:56

Description

See the title. The error log is as follows:

2015-07-30 17:32:27,963 INFO: org.apache.tajo.worker.TaskImpl (run(459)) - ta_1438245135274_0001_000001_000000_00 completed. Worker's task counter - total:1, succeeded: 0, killed: 0, failed: 1
2015-07-30 17:32:27,964 ERROR: org.apache.tajo.worker.TaskContainer (run(70)) - org.apache.tajo.exception.UnimplementedException: not implemented feature: name(RECORD) is not supported.
java.lang.RuntimeException: org.apache.tajo.exception.UnimplementedException: not implemented feature: name(RECORD) is not supported.
	at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:408)
	at org.apache.tajo.worker.TaskContainer.run(TaskContainer.java:65)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:744)
Caused by: org.apache.tajo.exception.UnimplementedException: not implemented feature: name(RECORD) is not supported.
	at org.apache.tajo.storage.json.JsonLineSerializer.serialize(JsonLineSerializer.java:114)
	at org.apache.tajo.storage.text.DelimitedTextFile$DelimitedTextFileAppender.addTuple(DelimitedTextFile.java:186)
	at org.apache.tajo.engine.planner.physical.StoreTableExec.next(StoreTableExec.java:129)
	at org.apache.tajo.worker.TaskImpl.run(TaskImpl.java:399)
	... 6 more
2015-07-30 17:32:27,965 ERROR: org.apache.tajo.querymaster.TaskAttempt (transition(423)) - ta_1438245135274_0001_000001_000000_00 FROM 192.168.10.102 >> not implemented feature: name(RECORD) is not supported.
2015-07-30 17:32:27,965 INFO: org.apache.tajo.worker.TaskAttemptContext (setState(137)) - Query status of ta_1438245135274_0001_000001_000000_00 is changed to TA_FAILED
2015-07-30 17:32:27,966 INFO: org.apache.tajo.querymaster.Task (transition(663)) - ====================================================================================
2015-07-30 17:32:27,966 INFO: org.apache.tajo.querymaster.Task (transition(664)) - >>> Task Failed: ta_1438245135274_0001_000001_000000_00, retry:true, attempts:1 <<<
2015-07-30 17:32:27,966 INFO: org.apache.tajo.querymaster.Task (transition(666)) - ====================================================================================
2015-07-30 17:32:27,967 ERROR: org.apache.tajo.querymaster.TaskAttempt (handle(439)) - Can't handle this event at current state of ta_1438245135274_0001_000001_000000_00), eventType:TA_FATAL_ERROR, oldState:TA_FAILED, nextState:TA_FAILED
org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_FATAL_ERROR at TA_FAILED
	at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
	at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
	at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
	at org.apache.tajo.querymaster.TaskAttempt.handle(TaskAttempt.java:437)
	at org.apache.tajo.querymaster.QueryMasterTask$TaskAttemptEventDispatcher.handle(QueryMasterTask.java:236)
	at org.apache.tajo.querymaster.QueryMasterTask$TaskAttemptEventDispatcher.handle(QueryMasterTask.java:229)
	at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
	at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
	at java.lang.Thread.run(Thread.java:744)
2015-07-30 17:32:27,969 INFO: org.apache.tajo.querymaster.DefaultTaskScheduler (stop(158)) - Task Scheduler stopped
2015-07-30 17:32:27,969 INFO: org.apache.tajo.querymaster.DefaultTaskScheduler (run(139)) - TaskScheduler schedulingThread stopped

Issue Links

• requires: TAJO-1725 TaskContainer can hang when RuntimeException occurs in TaskImpl.