[HUDI-3974] Fix upgrade step wrt precombine field - ASF JIRA

Attach files

Attach Screenshot

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Blocker
Resolution: Invalid
Affects Version/s: None
Fix Version/s: 0.11.0
Component/s: None
Labels:
- pull-request-available

Story Points:
1

Description

This is related to ~~HUDI-3972~~

Details about the bug: if someone is not explicitly setting preCombine field, hudi still falls backs to the default value of "ts" and this goes into hoodie.properties as well. So, when reading MOR table, since we are projecting just the required columns, we also add preCombine field to it and if the original table does not have "ts", basic read fails (even if there are no log files, but just base files in MOR).

For the upgrade step, we need to update the preCombine field in the hoodie.properties. If the user does not set the preCombine field, it should be removed from the hoodie.properties; if the users sets the field, it should be updated in the hoodie.properties.