[HIVE-15437] avro tables join fails when - tbl join tbl_postfix

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.2.0
• Assignee: Yongzhi Chen
• Reporter: Yongzhi Chen
• Created: 15/Dec/16 14:36
• Updated: 26/Jul/17 00:03
• Resolved: 19/Dec/16 16:42

Description

The following queries return good results:
select * from table1 where col1=key1;
select * from table1_1 where col1=key1;
When join them together, it gets following error:

Caused by: java.io.IOException: org.apache.avro.AvroTypeException: Found long, expecting union
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121) ~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77) ~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:116) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.CombineHiveRecordReader.doNext(CombineHiveRecordReader.java:43) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116) ~[hive-exec-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.doNextWithExceptionHandler(HadoopShimsSecure.java:229) ~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]
        at org.apache.hadoop.hive.shims.HadoopShimsSecure$CombineFileRecordReader.next(HadoopShimsSecure.java:141) ~[hive-shims-common-2.2.0-SNAPSHOT.jar:2.2.0-SNAPSHOT]

The two avro tables both is defined by using avro schema, and the first table's name is the second table name's prefix.

Note that this happens when single map is reading input from both tables. If map-join is used with MR execution engine, or if Tez execution engine is used, this issue is not seen.

Issue Links

• links to: GitHub Pull Request #127