[HIVE-26654] Test with the TPC-DS benchmark

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 4.0.0-alpha-2
• Labels: hive-4.0.0-must
• Assignee: Unassigned
• Reporter: Sungwoo Park
• Created: 21/Oct/22 02:15
• Updated: 06/Apr/24 09:34

Description

This Jira reports the result of running system tests using the TPC-DS benchmark. The test scenario is:

1) create a database consisting of external tables from a 100GB or 1TB TPC-DS text dataset
2) load a database consisting of ORC tables
3) compute column statistics
4) run TPC-DS queries
5) check the results for correctness

For step 5), we will compare the results against Hive 3 (which has been tested against SparkSQL and Presto). We use Hive on Tez.

Sub-Tasks

• TPC-DS query 2 fails with java.lang.RuntimeException: cannot find field _col0 from []: Closed
• VectorUDAFBloomFilterMerge should take care of safe batch handling when working in parallel: Closed
• Compilation fails due to equivalence mapping violation when CTE materialization is enabled: Resolved
• TPC-DS query 16, 69, 94 return wrong results.: Resolved
• TPC-DS query 71 returns wrong results: Resolved
• SharedWorkOptimizer merges TableScan operators that have different DPP parents: Resolved
• Wrong results when shared work optimizer merges TS operator with different DPP edges: Closed
• ParallelEdgeFixer adds redundant reduce sink operators: Patch Available
• ParallelEdgeFixer inserts misconfigured operator and does not connect it in Tez DAG: Closed
• VectorizedMapJoin returns wrong result for TPC-DS query 97: Closed