Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-5641

Streamline Advanced Schema Evolution flow

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 0.13.0
    • 0.14.0
    • None
    • None

    Description

      Currently, Schema Evolution not always applied consistently and sometimes is re-applied multiple times causing issues for HoodieSparkRecord implementations (that is optimized to reuse underlying buffer):

      1. HoodieMergeHelper would apply SE transformer, then
      2. HoodieMergeHandle would run rewriteRecordWithNewSchema again

      Attachments

        Issue Links

          duplicates

          Bug - A problem which impairs or prevents the functions of the product. HUDI-5633 Fixing HoodieSparkRecord performance bottlenecks

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              alexey.kudinkin Alexey Kudinkin
              alexey.kudinkin Alexey Kudinkin
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: