[TOREE-425] sparkContext lazy initiation causes some issues when Toree is running on Yarn Cluster mode

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Invalid
• Affects Version/s: 0.2.0
• Fix Version/s: 0.2.0
• Component/s: Kernel
• Assignee: Luciano Resende
• Reporter: Luciano Resende
• Created: 16/Jul/17 23:20
• Updated: 03/Aug/17 19:13
• Resolved: 03/Aug/17 19:13

Description

Kernels running in yarn-cluster mode (when launched via spark-submit) must initialize a SparkContext in order for the Spark Yarn code to register the application as RUNNING: https://github.com/apache/spark/blob/3d4d11a80fe8953d48d8bfac2ce112e37d38dc90/resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala#L405

Issue Links

• links to: GitHub Pull Request #128