[HIVE-18652] Print Spark metrics on console

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Spark
• Target Version/s: 3.0.0
• Assignee: Sahil Takiar
• Reporter: Sahil Takiar
• Created: 08/Feb/18 02:02
• Updated: 17/Nov/22 08:55
• Resolved: 04/Jun/18 18:38

Description

For Hive-on-MR, each MR job launched prints out some stats about the job:

INFO  : 2018-02-07 17:51:11,218 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-02-07 17:51:18,396 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.87 sec
INFO  : 2018-02-07 17:51:25,742 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.34 sec
INFO  : MapReduce Total cumulative CPU time: 4 seconds 340 msec
INFO  : Ended Job = job_1517865654989_0004
INFO  : MapReduce Jobs Launched:
INFO  : Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 4.34 sec   HDFS Read: 7353 HDFS Write: 151 SUCCESS
INFO  : Total MapReduce CPU Time Spent: 4 seconds 340 msec

We should do the same for Hive-on-Spark.

Issue Links

• links to: RB