Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20205

CDC source shouldn't send UPDATE_BEFORE messages if the downstream doesn't need it

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.12.0
    • Table SQL / Planner
    • None

    Description

      Currently, the CDC source will always generate UPDATE_BEFORE and UPDATE_AFTER messages for update. However, many downstream operators don't need the UDPATE_BEFORE, e.g. temporal join and upsert sink. Currently, debezium-cdc-source => upsert-kafka will generate lots of tombstone messages, which is quite not efficient.

      Attachments

        Issue Links

          is fixed by

          Sub-task - The sub-task of the issue FLINK-19795 Fix Flink SQL throws exception when changelog source contains duplicate change events

          • Major - Major loss of function.
          • Closed

          Activity

            People

              leonard Leonard Xu
              jark Jark Wu
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: