[HIVE-15680] Incorrect results when hive.optimize.index.filter=true and same ORC table is referenced twice in query

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0, 2.2.0
• Fix Version/s: 2.4.0, 3.0.0
• Assignee: Anthony Hsu
• Reporter: Anthony Hsu
• Created: 20/Jan/17 17:21
• Updated: 11/Mar/19 11:22
• Resolved: 27/Jan/18 00:15

Description

To repro:

set hive.optimize.index.filter=true;

create table test_table(number int) stored as ORC;

-- Two insertions will create two files, with one stripe each
insert into table test_table VALUES (1);
insert into table test_table VALUES (2);

-- This should and does return 2 records
select * from test_table;

-- These should and do each return 1 record
select * from test_table where number = 1;
select * from test_table where number = 2;

-- This should return 2 records but only returns 1 record
select * from test_table where number = 1
union all
select * from test_table where number = 2;

What's happening is only the last predicate is being pushed down.

Issue Links

• causes: HIVE-18873 Skipping predicate pushdown for MR silently at HiveInputFormat can cause storage handlers to produce erroneous result
• incorporates: HIVE-21413 Bad result using UNION with same table
• is related to: HIVE-18802 Incorrect results when referencing same Accumulo table multiple times in one query
• is related to: HIVE-12762 Common join on parquet tables returns incorrect result when hive.optimize.index.filter set to true
• is related to: HIVE-18695 fix TestAccumuloCliDriver.testCliDriver[accumulo_queries]
• links to: Review Request