[SPARK-30550] Random pyspark-shell applications being generated

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 2.3.2, 2.4.4
• Component/s: PySpark, Spark Shell, Spark Submit, Web UI
• Assignee: Unassigned
• Reporter: Ram
• Created: 17/Jan/20 10:12
• Updated: 12/Dec/22 18:11
• Resolved: 23/Jan/20 02:17

Description

 

When we submit a particular spark job, this happens. Not sure from where these pyspark-shell applications get generated, but they persist for like 5s and gets killed. We're not able to figure put why this happens