[SPARK-27045] SQL tab in UI shows actual SQL instead of callsite in case of SparkSQLDriver

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.2, 2.3.3, 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL, Web UI
• Assignee: Ajith S
• Reporter: Ajith S
• Created: 04/Mar/19 12:51
• Updated: 12/Mar/19 23:34
• Resolved: 12/Mar/19 23:15

Description

When we run sql in spark via SparkSQLDriver (thrift server, spark-sql), SQL string is siet via setJobDescription. the SparkUI SQL tab must show SQL instead of stacktrace in case setJobDescription is set which is more useful to end user. Instead it currently shows in description column the callsite shortform which is less useful

 Actual:

 

Expected:

Issue Links

• links to: GitHub Pull Request #23958