[HUDI-1932] Hive Sync should not always update last_commit_time_sync

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 0.10.0
• Component/s: hive
• Labels: AWS pull-request-available
• Assignee: Raymond Xu
• Reporter: Raymond Xu
• Created: 25/May/21 16:28
• Updated: 08/Dec/21 07:00

Description

In Hive Sync 

org.apache.hudi.hive.HiveSyncTool#syncHoodieTable()

it includes schema update and partition update.

Regardless of having update or not, the logic is to always update the table with a new timestamp for the last_commit_time_sync property.

This will result in having new versions in AWS Glue tables for example, which will eventually lead to exceeding table version limit.

Issue Links

• links to: GitHub Pull Request #3053