[HUDI-3855] Hudi's metadata field "_hoodie_file_name" not updated in MergeHandle

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Component/s: writer-core
• Labels: pull-request-available
• Story Points: 1
• Epic Link: Hudi-BigQuery
• Assignee: Alexey Kudinkin
• Reporter: Alexey Kudinkin

Reviewers

Raymond Xu, sivabalan narayanan

• Created: 12/Apr/22 01:00
• Updated: 23/May/22 00:31
• Resolved: 13/Apr/22 01:02

Description

As was reported by the user here: 
https://github.com/apache/hudi/issues/5231

 

Quoting:

So i was able to reproduce behavior that you're seeing and it turns out to be that _hoodie_file_name is simply not updated during Commit 3, meaning that during C3, all records are copied from latest base-file of the file-group into new latest base-file (in your most recent experiment it's c872d135-bf8f-4c5e-9eee-6347635c32d3-0_0-21-22_20220406182741563.parquet) but it doesn't update the _hoodie_file_name field which is kept pointing at the old file.

Issue Links

• causes: HUDI-3921 Reconcile schema evolution logic with base file re-writing
• links to: GitHub Pull Request #5296