[FLINK-20348] Make "schema-registry.subject" optional for Kafka sink with avro-confluent format

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile), Table SQL / Ecosystem
• Labels: pull-request-available sprint
• Assignee: zhuxiaoshang
• Reporter: Jark Wu
• Created: 25/Nov/20 15:09
• Updated: 28/May/21 07:09
• Resolved: 15/Jan/21 14:42

Description

Currently, configuration "schema-registry.subject" in avro-confluent format is required by sink. However, this is quite verbose set it manually. By default, it can be to set to <topic_name>-key and <topic_name>-value if it works with kafka or upsert-kafka connector. This can also makes 'avro-confluent' format to be more handy and works better with Kafka/Confluent ecosystem.

CREATE TABLE kafka_gmv (
  day_str STRING,
  gmv BIGINT,
  PRIMARY KEY (day_str) NOT ENFORCED
) WITH (
    'connector' = 'upsert-kafka',
    'topic' = 'kafka_gmv',
    'properties.bootstrap.servers' = 'localhost:9092',
    -- 'key.format' = 'raw',
    'key.format' = 'avro-confluent',
    'key.avro-confluent.schema-registry.url' = 'http://localhost:8181',
    'key.avro-confluent.schema-registry.subject' = 'kafka_gmv-key',
    'value.format' = 'avro-confluent',
    'value.avro-confluent.schema-registry.url' = 'http://localhost:8181',
    'value.avro-confluent.schema-registry.subject' = 'kafka_gmv-value'
);

Issue Links

• links to: GitHub Pull Request #14530