[SPARK-47503] Spark history sever fails to display query for cached JDBC relation named in quotes - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 3.2.1, 4.0.0
Fix Version/s: 4.0.0, 3.5.2, 3.4.3
Component/s: SQL
Labels:
- pull-request-available

Description

Spark history sever fails to display query for cached JDBC relation (or calculation derived from it) named in quotes

(Screenshot and generated history in attachments)

How to reproduce:

val ticketsDf = spark.read.jdbc("jdbc:postgresql://localhost:5432/demo", """ "test-schema".tickets """.trim, properties)
val bookingDf = spark.read.parquet("path/bookings")

ticketsDf.cache().count()

val resultDf = bookingDf.join(ticketsDf, Seq("book_ref"))

resultDf.write.mode(SaveMode.Overwrite).parquet("path/result")

So the problem is in SparkPlanGraphNode class which creates a dot node. When there is no metrics to display it simply returns tagged name and in this case name contains quotes which corrupts dot file.
Suggested solution is to escape name string