[HIVE-11097] HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 0.13.0, 0.13.1, 0.14.0, 1.0.0, 1.2.0
• Fix Version/s: 1.3.0, 2.1.0
• Component/s: File Formats

Environment

Hive 0.13.1, Hive 2.0.0, hadoop 2.4.1

• Assignee: Wan Chang
• Reporter: Wan Chang
• Created: 24/Jun/15 17:29
• Updated: 21/Jun/16 15:52
• Resolved: 05/Feb/16 22:16

Description

Say we have a sql as

create table if not exists test_orc_src (a int, b int, c int) stored as orc;
create table if not exists test_orc_src2 (a int, b int, d int) stored as orc;
insert overwrite table test_orc_src select 1,2,3 from src limit 1;
insert overwrite table test_orc_src2 select 1,2,4 from src limit 1;
set hive.auto.convert.join = false;
set hive.execution.engine=mr;
select
  tb.c
from test.test_orc_src tb
join (select * from test.test_orc_src2) tm
on tb.a = tm.a
where tb.b = 2

The correct result is 3 but it produced no result.

I find that in HiveInputFormat.pushProjectionsAndFilters

match = splitPath.startsWith(key) || splitPathWithNoSchema.startsWith(key);

It uses startsWith to combine aliases with path, so tm will match two alias in this case.

Issue Links

• relates to: HIVE-1903 Can't join HBase tables if one's name is the beginning of the other