[HIVE-20169] Print Final Rows Processed in MapOperator

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0, 4.0.0
• Fix Version/s: 3.2.0, 4.0.0-alpha-1
• Component/s: Operators
• Labels: newbie noob
• Assignee: Bharath Krishna
• Reporter: David Mollitor
• Created: 13/Jul/18 14:51
• Updated: 17/Nov/22 08:55
• Resolved: 02/Aug/18 17:29

Description

https://github.com/apache/hive/blob/ac6b2a3fb195916e22b2e5f465add2ffbcdc7430/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java#L573-L582

This class emits a log message every time it a certain number of records are processed, but it does not print a final count.

Overload the MapOperator class's closeOp method to print a final log message providing the total number of rows read by this mapper.

Issue Links

• is related to: HIVE-10078 Optionally allow logging of records processed in fixed intervals