[HIVE-18652] Print Spark metrics on console - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Bulk Copy Attachments

Bulk Move Attachments

Voters

Watch issue

Watchers

Convert to Issue

Move

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 4.0.0-alpha-1
Component/s: Spark
Labels:
None

Target Version/s:

3.0.0

Description

For Hive-on-MR, each MR job launched prints out some stats about the job:

INFO  : 2018-02-07 17:51:11,218 Stage-1 map = 0%,  reduce = 0%
INFO  : 2018-02-07 17:51:18,396 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.87 sec
INFO  : 2018-02-07 17:51:25,742 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 4.34 sec
INFO  : MapReduce Total cumulative CPU time: 4 seconds 340 msec
INFO  : Ended Job = job_1517865654989_0004
INFO  : MapReduce Jobs Launched:
INFO  : Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 4.34 sec   HDFS Read: 7353 HDFS Write: 151 SUCCESS
INFO  : Total MapReduce CPU Time Spent: 4 seconds 340 msec

We should do the same for Hive-on-Spark.