Uploaded image for project: 'Apache Hudi'
  1. Apache Hudi
  2. HUDI-4705

Support Write-on-compaction mode when query cdc on MOR tables

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • compaction, spark, table-service
    • None

    Description

      For the case that query cdc on MOR tables, the initial implementation use the `Write-on-indexing`  way to extract the cdc data by merging the base file and log files in-flight.

      This ticket wants to support the `Write-on-compaction` way to get the cdc data just by reading the persisted cdc files which are written at the compaction operation.

      Attachments

        Issue Links

          is part of

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. HUDI-3478 RFC-51: Change Data Capture for Hudi

          • Critical - Crashes, loss of data, severe memory leak.
          • Closed

          Activity

            People

              Unassigned Unassigned
              biyan900116@gmail.com Yann Byron
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: