[FLINK-19322] Specified parallelism in create table operation

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.12.0
• Component/s: Connectors / Common, Table SQL / API
• Labels: stale-major
• Assignee: Unassigned
• Reporter: YufeiLiu
• Created: 21/Sep/20 09:36
• Updated: 29/Apr/21 15:42
• Resolved: 29/Apr/21 15:42

Description

Flink SQL job only has global parallelism, and all operators share the same parallelism. But in many cases like using kafka connector, I want to set table parallelism separately to control resource usage.

• Source & Sink topic has different partition size.
• Consume multi topic with different throughput.
• Increase sink parallelism to insure write speed.

I think it would be better if we can specified parallelism property in create table operation, and the actual implementation depends on different connectors.

The create table operation may like this:

CREATE TABLE source (
  ...
) WITH (
  'connector' = 'kafka',
  'parallelism' = '2'
)

Issue Links

• is duplicated by: FLINK-18508 Dynamic source supports statistics and parallelism report