[SPARK-25131] Event logs missing applicationAttemptId for SparkListenerApplicationStart

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Affects Version/s: 2.3.1
• Component/s: Spark Core, YARN
• Assignee: Unassigned
• Reporter: Ajith S
• Created: 16/Aug/18 13:04
• Updated: 17/May/20 18:13
• Resolved: 20/Aug/18 02:04

Description

When master=yarn and deploy-mode=client, event logs do not contain applicationAttemptId for SparkListenerApplicationStart. This is caused at org.apache.spark.scheduler.cluster.YarnClientSchedulerBackend#start where we do bindToYarn(client.submitApplication(), None) which sets appAttemptId to None. We can however, get the appAttemptId after waitForApplication() and set it

Issue Links

• links to: [Github] Pull Request #22120 (ajithme)