Uploaded image for project: 'Apache Tez'
  1. Apache Tez
  2. TEZ-3186

teztask event problem when running repeated queries on LLAP

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Cannot Reproduce
    • None
    • None
    • None
    • None

    Description

      I am running multiple queries in a row against LLAP from CLI.
      I was running them by copy-pasting multiple lines of "source this.sql" and "source that.sql" into CLI.
      When I switched to running via hive -f all-queries.sql (could be a coincidence), one of the queries now fails towards the end with an error like this:

      2016-03-23 21:57:35,531 [INFO] [TaskSchedulerEventHandlerThread] |tezplugins.LlapTaskSchedulerService|: Ignoring deallocate request for task attempt_1455662455106_3046_5_00_000526_0 which hasn't been assigned to a container
      2016-03-23 21:57:35,531 [INFO] [TaskSchedulerEventHandlerThread] |rm.TaskSchedulerManager|: Task: attempt_1455662455106_3046_5_00_000526_0 has no container assignment in the scheduler
      2016-03-23 21:57:35,533 [ERROR] [Dispatcher thread {Central}] |impl.TaskAttemptImpl|: Can't handle this event at current state for attempt_1455662455106_3046_5_00_000006_1
      org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: TA_TEZ_EVENT_UPDATE at KILL_IN_PROGRESS
              at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
              at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
              at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
              at org.apache.tez.dag.app.dag.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:795)
              at org.apache.tez.dag.app.dag.impl.TaskAttemptImpl.handle(TaskAttemptImpl.java:120)
              at org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher.handle(DAGAppMaster.java:2202)
              at org.apache.tez.dag.app.DAGAppMaster$TaskAttemptEventDispatcher.handle(DAGAppMaster.java:2187)
              at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
              at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
              at java.lang.Thread.run(Thread.java:745)
      2016-03-23 21:57:35,537 [INFO] [Dispatcher thread {Central}] |history.HistoryEventHandler|: [HISTORY][DAG:dag_1455662455106_3046_5][Event:TASK_FINISHED]: vertexName=Map 1, taskId=task_1455662455106_3046_5_00_000527, startTime=1458784644802, finishTime=1458784655537, timeTaken=10735, status=KILLED, successfulAttemptID=null, diagnostics=Killing tasks in vertex: vertex_1455662455106_3046_5_00 [Map 1] due to trigger: OWN_TASK_FAILURE, counters=Counters: 0
      

      This is on master.

      Attachments

        Issue Links

          Activity

            People

              sseth Siddharth Seth
              sershe Sergey Shelukhin
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: