[SPARK-13670] spark-class doesn't bubble up error from launcher command

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: Spark Submit
• Assignee: Marcelo Masiero Vanzin
• Reporter: Mark Grover
• Created: 04/Mar/16 07:06
• Updated: 10/May/16 17:34
• Resolved: 10/May/16 17:34

Description

There's a particular snippet in spark-class here that runs the spark-launcher code in a subshell.

# The launcher library will print arguments separated by a NULL character, to allow arguments with
# characters that would be otherwise interpreted by the shell. Read that in a while loop, populating
# an array that will be used to exec the final command.
CMD=()
while IFS= read -d '' -r ARG; do
  CMD+=("$ARG")
done < <("$RUNNER" -cp "$LAUNCH_CLASSPATH" org.apache.spark.launcher.Main "$@")

The problem is that the if the launcher Main fails, this code still still returns success and continues, even though the top level script is marked set -e. This is because the launcher.Main is run within a subshell.

Issue Links

• links to: [Github] Pull Request #12910 (vanzin)