Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-3348

HoodieRealtimeFileSplit losing info when serialized/deserialized

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • None
    • 0.11.0
    • None
    • 0.5

    Description

      https://github.com/apache/hudi/pull/3865 added new field deltaLogFiles

       

      But it didn't corresponding modify the readFields, write routines that Hive is using to ser/de splits, meaning that essentially this info is lost upon passing the data b/w executors.

      Attachments

        Issue Links

        Discovered while testing

        Improvement - An improvement or enhancement to an existing feature or task. HUDI-2762 Ensure hive can query insert only logs in MOR

        • Critical - Crashes, loss of data, severe memory leak.
        • Reopened
        is a child of

        Task - A task that needs to be done. HUDI-3280 Clean up unused/deprecated methods

        • Blocker - Blocks development and/or testing work, production could not run
        • Closed
        links to

        Web Link GitHub Pull Request #4951

        Activity
          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            xushiyan Shiyan Xu
            alexey.kudinkin Alexey Kudinkin
            Alexey Kudinkin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Agile

                Completed Sprint:
                Hudi-Sprint-Mar-01 ended 08/Mar/22
                View on Board

                Slack

                  Issue deployment