[FLINK-31060] Release Testing: Verify FLINK-30542 Support adaptive local hash aggregate in runtime

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.17.0
• Fix Version/s: 1.17.0
• Component/s: Table SQL / Runtime
• Assignee: dalongliu
• Reporter: Yunhong Zheng
• Created: 14/Feb/23 08:38
• Updated: 21/Feb/23 04:36
• Resolved: 21/Feb/23 04:36

Description

This issue aims to verify FLINK-30542: Support adaptive local hash aggregate in runtime.

Adaptive local hash aggregation is an optimization of local hash aggregation, which can adaptively determine whether to continue to do local hash aggregation according to the distinct value rate of sampling data. If distinct value rate bigger than defined threshold (see parameter: 'table.exec.local-hash-agg.adaptive.distinct-value-rate-threshold'), we will stop aggregating and just send the input data to the downstream after a simple projection. Otherwise, we will continue to do aggregation.

We can verify it in SQL client after we build the flink-dist package.

1. Create a source table firstly. (Note: the source table need have different degree of aggregation, means the distinct count can be controlled by source connector, we recommend to modify dataGen table source to produce different data with different distinct row number).
2. Verify the result with different distinct value rate. (See: table.exec.local-hash-agg.adaptive.distinct-value-rate-threshold)
3. Check the log in 'TM' to see whether the adaptive local hash aggregate works.

If you meet any problems, it's welcome to ping me directly.