[FLINK-36509] Fix 'Unsupported bucket mode: GLOBAL_DYNAMIC' error in Paimon Pipeline Sink.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: cdc-3.2.0
• Fix Version/s: cdc-3.3.0, cdc-3.2.1
• Component/s: Flink CDC
• Labels: pull-request-available
• Assignee: Yanquan Lv
• Reporter: Yanquan Lv
• Created: 11/Oct/24 03:46
• Updated: 20/Nov/24 11:06
• Resolved: 20/Nov/24 02:24

Description

When using Paimon as pipeline sink, If a field is added to the partition field but not to the primary key field in transform, we will met Unsupported bucket mode: GLOBAL_DYNAMIC error in Paimon sink.

How to reproduct:

 

source:
  type: mysql
  name: MySQL Source
  hostname: 127.0.0.1
  port: 3306
  username: admin
  password: pass
  tables: bdb.user_table
  server-id: 5401-5404

transform:
  - source-table: bdb.user_table
    partition-keys: product_name

sink:
  type: paimon
  name: Paimon Sink
  catalog.properties.metastore: filesystem
  catalog.properties.warehouse: /path/warehouse

pipeline:
  name: MySQL to Paimon Pipeline
  parallelism: 2

 

 

Issue Links

• links to: GitHub Pull Request #3726
• links to: https://github.com/apache/flink-cdc/pull/3641