[SPARK-24243] Expose exceptions from InProcessAppHandle

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 3.0.0
• Component/s: Spark Submit
• Assignee: Sahil Takiar
• Reporter: Sahil Takiar
• Created: 10/May/18 12:42
• Updated: 07/Dec/18 18:35
• Resolved: 07/Dec/18 18:35

Description

InProcessAppHandle runs SparkSubmit in a dedicated thread, any exceptions thrown are logged and then the state is set to FAILED. It would be nice to expose the Throwable object to the application rather than logging it and dropping it. Applications may want to manipulate the underlying Throwable / control its logging at a finer granularity. For example, the app might want to call Throwables.getRootCause(throwable).getMessage() and expose the message to the app users.

Issue Links

• relates to: HIVE-18533 Add option to use InProcessLauncher to submit spark jobs
• links to: [Github] Pull Request #21849 (sahilTakiar)
• links to: [Github] Pull Request #23221 (vanzin)
• links to: [Github] Pull Request #23221 (vanzin)