[SPARK-22794] Spark Job failed, but the state is succeeded in Yarn Web

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.2.1
• Component/s: Spark Core, YARN
• Assignee: Unassigned
• Reporter: KaiXinXIaoLei
• Created: 15/Dec/17 07:52
• Updated: 17/May/20 18:14
• Resolved: 19/Dec/17 15:01

Description

I run a job in yarn mode, the job is failed:

17/12/15 11:55:16 INFO SharedState: Warehouse path is '/apps/hive/warehouse'.
Exception in thread "main" org.apache.spark.sql.AnalysisException: Path does not exist: hdfs://node1.huleilei.h3c.com:8020/user/hdfs/sss;
	at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$14.apply(DataSource.scala:382)
	at org.apache.spark.sql.execution.datasources.DataSource$$anonfun$14.apply(DataSource.scala:370)

but in the yarn web, the job state is succeeded.

Issue Links

• relates to: SPARK-15283 YARN Application should get FAILED state if driver is failed in yarn-client mode