[TWILL-217] AppMaster launcher should include eventHandler dependencies and nothing else from application

• Type: Improvement
• Status: In Progress
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Component/s: yarn
• Assignee: Martin Serrano
• Reporter: Martin Serrano
• Created: 11/Feb/17 22:16
• Updated: 24/Feb/17 18:09

Description

Currently the launcher for the appmaster includes the application.jar libraries. This is to support user code that adds an EventHandler. The application may have many dependencies and including them in the appmaster classpath can lead to otherwise inaddressable incompatibilities.

In my case, something in my application's large dependency graph was interfering with the Kafka server operation. I was not able to determine what it was but tweaking the appmaster loader to not include my application jars fixed the issue.

Instead the bundler that creates the twill.jar should include the EventHandler extension (if any) as an explicit dependency. In this way, only the jars needed to support the event handler will be on the twill classpath.