[IMPALA-10365] test_page_index failed in exhaustive release build

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: Backend
• Labels: broken-build correctness
• Epic Color: ghx-label-2
• Assignee: Zoltán Borók-Nagy
• Reporter: Zoltán Borók-Nagy
• Created: 30/Nov/20 16:36
• Updated: 17/Feb/21 18:01
• Resolved: 17/Feb/21 18:01

Description

query_test.test_parquet_stats.TestParquetStats.test_page_index failed in exhaustive release build.

Stack Trace

query_test/test_parquet_stats.py:98: in test_page_index
 unique_database)
common/impala_test_suite.py:693: in run_test_case
 self.__verify_results_and_errors(vector, test_section, result, use_db)
common/impala_test_suite.py:529: in __verify_results_and_errors
 replace_filenames_with_placeholder)
common/test_result_verifier.py:456: in verify_raw_results
 VERIFIER_MAP[verifier](expected, actual)
common/test_result_verifier.py:278: in verify_query_result_is_equal
 assert expected_results == actual_results
E assert Comparing QueryTestResults (expected vs actual):
E 'AIR','1-URGENT',4 != 'AIR','1-URGENT',3
E 'AIR','2-HIGH',3 != 'AIR','2-HIGH',2
E 'AIR','3-MEDIUM',4 == 'AIR','3-MEDIUM',4
E 'AIR','4-NOT SPECIFIED',2 == 'AIR','4-NOT SPECIFIED',2
E 'AIR','5-LOW',1 == 'AIR','5-LOW',1
E 'FOB','1-URGENT',3 == 'FOB','1-URGENT',3
E 'FOB','2-HIGH',1 != 'FOB','3-MEDIUM',2
E 'FOB','3-MEDIUM',2 != 'FOB','4-NOT SPECIFIED',3
E 'FOB','4-NOT SPECIFIED',3 != 'FOB','5-LOW',1
E 'FOB','5-LOW',2 != 'MAIL','2-HIGH',2
E 'MAIL','2-HIGH',3 != 'MAIL','3-MEDIUM',3
E 'MAIL','3-MEDIUM',3 != 'MAIL','4-NOT SPECIFIED',2
E 'MAIL','4-NOT SPECIFIED',3 != 'MAIL','5-LOW',1
E 'MAIL','5-LOW',1 != 'RAIL','2-HIGH',1
E 'RAIL','2-HIGH',1 != 'RAIL','3-MEDIUM',2
E 'RAIL','3-MEDIUM',2 != 'RAIL','4-NOT SPECIFIED',1
E 'RAIL','4-NOT SPECIFIED',1 != 'RAIL','5-LOW',1
E 'RAIL','5-LOW',1 != 'REG AIR','2-HIGH',3
E 'REG AIR','2-HIGH',4 != 'REG AIR','3-MEDIUM',1
E 'REG AIR','3-MEDIUM',2 != 'REG AIR','5-LOW',2
E 'REG AIR','5-LOW',2 != 'SHIP','1-URGENT',3
E 'SHIP','1-URGENT',4 != 'SHIP','2-HIGH',2
E 'SHIP','2-HIGH',2 != 'SHIP','4-NOT SPECIFIED',2
E 'SHIP','4-NOT SPECIFIED',2 != 'TRUCK','3-MEDIUM',2
E 'TRUCK','3-MEDIUM',2 != 'TRUCK','5-LOW',2
E 'TRUCK','5-LOW',3 != None
E Number of rows returned (expected vs actual): 26 != 25

After adjusting the lines:

E 'AIR','1-URGENT',4 != 'AIR','1-URGENT',3
E 'AIR','2-HIGH',3 != 'AIR','2-HIGH',2
E 'AIR','3-MEDIUM',4 == 'AIR','3-MEDIUM',4
E 'AIR','4-NOT SPECIFIED',2 == 'AIR','4-NOT SPECIFIED',2
E 'AIR','5-LOW',1 == 'AIR','5-LOW',1
E 'FOB','1-URGENT',3 == 'FOB','1-URGENT',3
E 'FOB','2-HIGH',1 != None
E 'FOB','3-MEDIUM',2 == 'FOB','3-MEDIUM',2
E 'FOB','4-NOT SPECIFIED',3 == 'FOB','4-NOT SPECIFIED',3
E 'FOB','5-LOW',2 != 'FOB','5-LOW',1
E 'MAIL','2-HIGH',3 != 'MAIL','2-HIGH',2
E 'MAIL','3-MEDIUM',3 == 'MAIL','3-MEDIUM',3
E 'MAIL','4-NOT SPECIFIED',3 != 'MAIL','4-NOT SPECIFIED',2
E 'MAIL','5-LOW',1 == 'MAIL','5-LOW',1
E 'RAIL','2-HIGH',1 == 'RAIL','2-HIGH',1
E 'RAIL','3-MEDIUM',2 == 'RAIL','3-MEDIUM',2
E 'RAIL','4-NOT SPECIFIED',1 == 'RAIL','4-NOT SPECIFIED',1
E 'RAIL','5-LOW',1 == 'RAIL','5-LOW',1
E 'REG AIR','2-HIGH',4 != 'REG AIR','2-HIGH',3
E 'REG AIR','3-MEDIUM',2 != 'REG AIR','3-MEDIUM',1
E 'REG AIR','5-LOW',2 == 'REG AIR','5-LOW',2
E 'SHIP','1-URGENT',4 != 'SHIP','1-URGENT',3
E 'SHIP','2-HIGH',2 == 'SHIP','2-HIGH',2
E 'SHIP','4-NOT SPECIFIED',2 == 'SHIP','4-NOT SPECIFIED',2
E 'TRUCK','3-MEDIUM',2 == 'TRUCK','3-MEDIUM',2
E 'TRUCK','5-LOW',3 != 'TRUCK','5-LOW',2

The query was:

select l_shipmode, o_orderpriority, count(*)
 from tpch_nested_parquet.customer.c_orders o, o.o_lineitems l
 where l_receiptdate < '1992-01-10'
 group by l_shipmode, o_orderpriority

Nested tables are written by Hive, so it must have been a reader-side problem that also passed 
HdfsParquetScanner::CheckPageFiltering().

Issue Links

• duplicates: IMPALA-10501 Hit DCHECK in parquet-column-readers.cc: def_levels_.CacheRemaining() <= num_buffered_values_