[TEZ-940] TaskSchedulerAppCallbackWrapper leaks memory via a CompletionService

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.4.0
• Fix Version/s: 0.4.0
• Assignee: Siddharth Seth
• Reporter: Gopal Vijayaraghavan
• Created: 17/Mar/14 23:30
• Updated: 30/Mar/14 08:01
• Resolved: 18/Mar/14 17:47

Description

The ExecutorService in TaskSchedulerAppCallbackWrapper uses an ExecutorCompletionService to keep track of submitted tasks and their results.

The difference between the Executor and Completion services is that without calling a take()/poll(), the latter leaks FutureTask objects which are held in memory until shutdown.

This results in a leak, from TaskAllocatedCallable -> TezVertexImpl -> TezDagImpl -> Configuration.