[SPARK-26737] Executor/Task STDERR & STDOUT log urls are not correct in Yarn deployment mode

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Spark Core, Web UI, YARN
• Assignee: Jungtaek Lim
• Reporter: Devaraj Kavali
• Created: 26/Jan/19 07:05
• Updated: 17/May/20 18:14
• Resolved: 30/Jan/19 19:55

Description

Base of the STDERR & STDOUT log urls are generating like these which is also including key,

http://ip:8042/node/containerlogs/container_1544212645385_0252_01_000001/(SPARK_USER, devaraj)

http://ip:8042/node/containerlogs/container_1544212645385_0252_01_000001/(USER, devaraj)

Instead of

http://ip:8042/node/containerlogs/container_1544212645385_0251_01_000002/devaraj 

Issue Links

• is related to: SPARK-22404 Provide an option to use unmanaged AM in yarn-client mode
• links to: GitHub Pull Request #23659