[TEZ-1661] LocalTaskScheduler hangs when shutdown

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.5.0
• Fix Version/s: 0.7.0, 0.6.1

Environment

Local Mode

• Target Version/s: 0.7.0
• Assignee: Jeff Zhang
• Reporter: Oleg Zhurakousky
• Created: 13/Oct/14 23:19
• Updated: 31/Jul/20 09:56
• Resolved: 16/Jan/15 10:59

Description

LocalTaskScheduler hangs on 'take' from the 'taskRequestQueue ' when TezClient shuts down (e.g., TezClient.stop).
Below is jstack output observed when running in Tez local mode:

"Thread-53" prio=5 tid=0x00007fc876d8f800 nid=0xac07 runnable [0x000000011df90000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.Throwable.fillInStackTrace(Native Method)
        at java.lang.Throwable.fillInStackTrace(Throwable.java:783)
        - locked <0x00000007b6ce60a0> (a java.lang.InterruptedException)
        at java.lang.Throwable.<init>(Throwable.java:250)
        at java.lang.Exception.<init>(Exception.java:54)
        at java.lang.InterruptedException.<init>(InterruptedException.java:57)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1219)
        at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:340)
        at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:535)
        at org.apache.tez.dag.app.rm.LocalTaskSchedulerService$AsyncDelegateRequestHandler.processRequest(LocalTaskSchedulerService.java:310)
        at org.apache.tez.dag.app.rm.LocalTaskSchedulerService$AsyncDelegateRequestHandler.run(LocalTaskSchedulerService.java:304)
        at java.lang.Thread.run(Thread.java:745)