[HIVE-24637] Make Tez progress log interval configurable

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0, 3.1.2, 4.0.0
• Fix Version/s: 4.0.0-alpha-1
• Component/s: Clients
• Labels: pull-request-available
• Assignee: Johan Gustavsson
• Reporter: Johan Gustavsson
• Created: 14/Jan/21 21:43
• Updated: 17/Nov/22 08:48
• Resolved: 26/Jan/21 14:17

Description

In case of hive on MR we can configure how often the progress log is updated in the client side with the parameter https://github.com/apache/hive/blob/master/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L2041-L2050 while in case of Hive on Tez this value is hard coded here https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java#L42. The default value in Tez is also significantly shorter than that of MR (6 VS 3 S) meaning for longer queries the client log can get very long.

Issue Links

• links to: GitHub Pull Request #1870