[SPARK-5034] Spark on Yarn launch failure on HDInsight on Windows

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 1.1.0, 1.1.1, 1.2.0
• Component/s: Windows, YARN

Environment

Spark on Yarn within HDInsight on Windows Azure

• Assignee: Unassigned
• Reporter: Rice
• Created: 31/Dec/14 08:22
• Updated: 08/May/15 08:21
• Resolved: 08/May/15 08:21

Description

Windows Environment

I'm trying to run JavaSparkPi example on YARN with master = yarn-client but I have a problem.
It runs smoothly with submitting application, first container for Application Master works too.
When job is starting and there are some tasks to do I'm getting this warning on console (I'm using windows cmd if this makes any difference):
WARN cluster.YarnClientClusterScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient memory
When I'm checking logs for container with Application Masters it is launching containers for executors "properly", then goes with:
INFO YarnAllocationHandler: Completed container container_1409217202587_0003_01_000002 (state: COMPLETE, exit status: 1)
INFO YarnAllocationHandler: Container marked as failed: container_1409217202587_0003_01_000002
And tries to re-launch them.

On failed container log there is only this:
Error: Could not find or load main class [e-mail]:63680.user.CoarseGrainedScheduler

Issue Links

• relates to: SPARK-5754 Spark AM not launching on Windows