[HIVE-24629] Invoke optional output committer in TezProcessor

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.0.0-alpha-1
• Labels: pull-request-available
• Assignee: Marton Bod
• Reporter: Marton Bod
• Created: 13/Jan/21 08:34
• Updated: 17/Nov/22 08:48
• Resolved: 18/Jan/21 09:48

Description

In order to enable Hive to write to Iceberg tables, we need to use an output committer which will fire at the end of each Tez task execution (commitTask) and the after the execution of each vertex (commitOutput/commitJob). This output committer will issue a commit containing the written-out data files to the Iceberg table, replacing its previous snapshot pointer with a new one.

Issue Links

• links to: GitHub Pull Request #1857