[HUDI-7552] Remove the suffix for MDT table service instants

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0-beta2, 1.0.0
• Component/s: core
• Labels: pull-request-available
• Story Points: 4
• Epic Link: metadata-table-improvements
• Assignee: Danny Chen
• Reporter: Danny Chen

Reviewers

Ethan Guo

• Created: 28/Mar/24 01:28
• Updated: 10/Jun/24 17:56
• Resolved: 11/Apr/24 02:21

Description

We wanna remove the very specific design for MDT so that it's behavior is in sync with the DT.

 

The criteria for simplification:

1. use the instant timestamp from DT to commit to the MDT as much as possible for any delta_commit on MDT.
2. for table sercives like cleaning, compaction and log_compaction, the timestamp is auto-generated.
3. avoid to trigger multiple commits to MDT for one DT action. 

The async index instant suffix is kept because there are some validation logic that needs special filtering on these instants, the suffix is kind of a "tag" for filtering. We should refacor that out in the future if we have a better solution.

Issue Links

• links to: GitHub Pull Request #10945