[FLINK-35473] FLIP-457: Improve Table/SQL Configuration for Flink 2.0

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.0
• Component/s: Table SQL / API
• Labels: pull-request-available

Release Note

As Apache Flink progresses to version 2.0, several table configuration options are being deprecated and replaced to improve user-friendliness and maintainability.

Deprecated Configuration Options

The following table configuration options are deprecated in this release and will be removed in Flink 2.0:

Deprecated Due to TPC Testing Irrelevance

These options were previously used for fine-tuning TPC testing but are no longer needed by the current Flink planner:

table.exec.range-sort.enabled
table.optimizer.rows-per-local-agg
table.optimizer.join.null-filter-threshold
table.optimizer.semi-anti-join.build-distinct.ndv-ratio
table.optimizer.shuffle-by-partial-key-enabled
table.optimizer.smj.remove-sort-enabled
table.optimizer.cnf-nodes-limit
Deprecated Due to Legacy Interface

These options were introduced for the now-obsolete FilterableTableSource interface:

table.optimizer.source.aggregate-pushdown-enabled
table.optimizer.source.predicate-pushdown-enabled
New and Updated Configuration Options
SQL Client Option
sql-client.display.max-column-width has been replaced with table.display.max-column-width.
Batch Execution Options

The following options have been moved from org.apache.flink.table.planner.codegen.agg.batch.HashAggCodeGenerator to org.apache.flink.table.api.config. and promoted to PublicEvolving:

table.exec.local-hash-agg.adaptive.enabled
table.exec.local-hash-agg.adaptive.sampling-threshold
table.exec.local-hash-agg.adaptive.distinct-value-rate-threshold
Lookup Hint Options

The following options have been moved from org.apache.flink.table.planner.hint.LookupJoinHintOptions to org.apache.flink.table.api.config.LookupJoinHintOptions and promoted to PublicEvolving:

table
async
output-mode
capacity
timeout
retry-predicate
retry-strategy
fixed-delay
max-attempts
Optimizer Options

The following options have been moved from org.apache.flink.table.planner.plan.optimize.RelNodeBlock to org.apache.flink.table.api.config.OptimizerConfigOptions and promoted to PublicEvolving:

table.optimizer.union-all-as-breakpoint-enabled
table.optimizer.reuse-optimize-block-with-digest-enabled
Aggregate Optimizer Option

The following option has been moved from org.apache.flink.table.planner.plan.rules.physical.stream.IncrementalAggregateRule to org.apache.flink.table.api.config.OptimizerConfigOptions and promoted to PublicEvolving:

table.optimizer.incremental-agg-enabled

• Assignee: Jane Chan
• Reporter: Jane Chan
• Created: 28/May/24 07:38
• Updated: 16/Jun/24 04:13
• Resolved: 11/Jun/24 14:09

Description

This is the parent task for FLIP-457.

Issue Links

• Testing discovered: FLINK-35569 SnapshotFileMergingCompatibilityITCase#testSwitchFromEnablingToDisablingFileMerging failed
• links to: GitHub Pull Request #24889
• mentioned in: Page Loading...

Sub-Tasks

• Remove deprecated Table/SQL configuration in 2.0: Open