Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3974

Fix upgrade step wrt precombine field

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Invalid
    • None
    • 0.11.0
    • None
    • 1

    Description

      This is related to HUDI-3972

      Details about the bug: if someone is not explicitly setting preCombine field, hudi still falls backs to the default value of "ts" and this goes into hoodie.properties as well. So, when reading MOR table, since we are projecting just the required columns, we also add preCombine field to it and if the original table does not have "ts", basic read fails (even if there are no log files, but just base files in MOR).

      For the upgrade step, we need to update the preCombine field in the hoodie.properties.  If the user does not set the preCombine field, it should be removed from the hoodie.properties; if the users sets the field, it should be updated in the hoodie.properties.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            guoyihua Ethan Guo (this is the old account; please use "yihua")
            guoyihua Ethan Guo (this is the old account; please use "yihua")
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment