[HIVE-10720] Pig using HCatLoader to access RCFile and perform join but get incorrect result.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.3.0
• Component/s: HCatalog
• Assignee: Aihua Xu
• Reporter: Aihua Xu
• Created: 15/May/15 15:03
• Updated: 28/Jan/17 18:28

Description

Create table tbl1 (c1 string, c2 string, key string, value string) stored as rcfile;
Create table tbl2 (key string, value string);
insert into tbl1 values('c1', 'c2', '1', 'value1');
insert into tbl2 values('1', 'value2');

Pig script:

tbl1 = LOAD 'tbl1' USING org.apache.hive.hcatalog.pig.HCatLoader();
tbl2 = LOAD 'tbl2' USING org.apache.hive.hcatalog.pig.HCatLoader();

src_tbl1 = FILTER tbl1 BY (key == '1');
prj_tbl1 = FOREACH src_tbl1 GENERATE
           c1 as c1,
           c2 as c2,
           key as tbl1_key;
           
src_tbl2 = FILTER tbl2 BY (key == '1');
prj_tbl2 = FOREACH src_tbl2 GENERATE
           key as tbl2_key;
           
result = JOIN prj_tbl1 BY (tbl1_key), prj_tbl2 BY (tbl2_key);
dump result;

You will see result "(,,1,1)" and we are missing c1 and c2 values.

Issue Links

• relates to: HIVE-10755 Rework on HIVE-5193 to enhance the column oriented table access

Sub-Tasks

• Revert HIVE-5193: Closed
• new Job() is deprecated. Replaced all with Job.getInstance() for Hcatalog: Patch Available
• Rework on HIVE-5193 to enhance the column oriented table access: Closed