[FLINK-15006] Add option to close shuffle when dynamic partition inserting

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.11.0
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Jingsong Lee
• Reporter: Jingsong Lee
• Created: 02/Dec/19 08:40
• Updated: 19/May/20 09:03
• Resolved: 26/Apr/20 07:33

Description

When partition values are rare or have skew, if we shuffle by dynamic partitions, will break the performance.

We can have an option to close shuffle in such cases:

‘connector.sink.shuffle-by-partition.enable’ = ...

Issue Links

• duplicates: FLINK-16818 Optimize data skew when flink write data to hive dynamic partition table
• relates to: FLINK-16818 Optimize data skew when flink write data to hive dynamic partition table
• links to: GitHub Pull Request #11853