[IMPALA-12601] Add a fully partitioned TPC-DS dataset for planner tests

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Impala 4.3.0
• Fix Version/s: Impala 4.4.0
• Component/s: Infrastructure
• Target Version: Impala 4.4.0
• Epic Color: ghx-label-3
• Assignee: Riza Suminto
• Reporter: Riza Suminto
• Created: 05/Dec/23 18:08
• Updated: 19/Dec/23 21:25
• Resolved: 19/Dec/23 21:24

Description

testdata/datasets/tpcds only have store_sales table fully-partitioned and leave other facts table unpartitioned. This is probably intended for faster data loading during test. However, this is not an accurate reflection of larger scale TPC-DS dataset where all facts tables are partitioned. Impala planner may change the detail of query plan if partition column exist.

Today, Impala can load a small scale, fully-partitioned TPC-DS dataset in parquet format to minicluster in reasonable speed. We should consider adding such test db and change PlannerTests to test against that fully-partitioned db instead of current tpcds/tpcds_parquet test db.

Issue Links

• causes: IMPALA-12655 PlannerTest.testProcessingCost seems flaky