[SPARK-22462] SQL metrics missing after foreach operation on dataframe

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.3.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: Juliusz Sompolski
• Created: 07/Nov/17 02:16
• Updated: 11/Nov/17 11:35
• Resolved: 11/Nov/17 11:34

Description

No SQL metrics are visible in the SQL tab of SparkUI when foreach is executed on the DataFrame.
e.g.

sql("select * from range(10)").collect()
sql("select * from range(10)").foreach(a => Unit)
sql("select * from range(10)").foreach(a => println(a))

See collect.png vs. foreach.png

Issue Links

• links to: [Github] Pull Request #19689 (viirya)