[HIVE-4742] A useless CAST makes Hive fail to create a VIEW based on an UNION

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0
• Fix Version/s: 0.12.0
• Assignee: Unassigned
• Reporter: Nicolas Lalevée
• Created: 17/Jun/13 09:02
• Updated: 03/Dec/13 19:11
• Resolved: 19/Aug/13 14:26

Description

I create programmatically a script to create a view which is a union of all kind of event I have. To keep things simple, data are just blindly cast as STRING. It used to work with Hive 0.10, not anymore with 0.11.

I tried to narrow down the simplest script. It seems that it only occurs if a least a view and an union is involved.

Here is a failing script:

CREATE TABLE Event1 (d STRING, userid BIGINT, eventData1 STRING);
CREATE TABLE Event2 (d STRING, userid BIGINT, eventData2 STRING);
CREATE VIEW AllEventsTest
  AS SELECT * FROM (
    SELECT 'Event1' AS eventType, map('d', CAST(`d` AS STRING)) AS eventData FROM Event1
    UNION ALL
    SELECT 'Event2' AS eventType, map('d', CAST(`d` AS STRING)) AS eventData FROM Event2
) d;

There are warnings in the logs:

o.a.h.h.q.parse.TypeCheckProcFactory - Invalid type entry TOK_STRING=null
o.a.h.h.q.parse.TypeCheckProcFactory - Invalid type entry TOK_STRING=null

And the error is:

FAILED: IllegalArgumentException replace op boundaries of <ReplaceOp@[@46,103:105='`d`',<26>,3:52]..[@46,103:105='`d`',<26>,3:52]:"`event1`.`d`"> overlap with previous <ReplaceOp@[@44,98:101='CAST',<48>,3:47]..[@51,116:116=')',<276>,3:65]:"`event1`.`d`">
10:52:51.024 [scoopMapredScheduler_Worker-9 ] [ERROR] org.apache.hadoop.hive.ql.Driver - FAILED: IllegalArgumentException replace op boundaries of <ReplaceOp@[@46,103:105='`d`',<26>,3:52]..[@46,103:105='`d`',<26>,3:52]:"`event1`.`d`"> overlap with previous <ReplaceOp@[@44,98:101='CAST',<48>,3:47]..[@51,116:116=')',<276>,3:65]:"`event1`.`d`">
java.lang.IllegalArgumentException: replace op boundaries of <ReplaceOp@[@46,103:105='`d`',<26>,3:52]..[@46,103:105='`d`',<26>,3:52]:"`event1`.`d`"> overlap with previous <ReplaceOp@[@44,98:101='CAST',<48>,3:47]..[@51,116:116=')',<276>,3:65]:"`event1`.`d`">
    at org.antlr.runtime.TokenRewriteStream.reduceToSingleOperationPerIndex(TokenRewriteStream.java:504)
    at org.antlr.runtime.TokenRewriteStream.toString(TokenRewriteStream.java:374)
    at org.antlr.runtime.TokenRewriteStream.toString(TokenRewriteStream.java:358)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.saveViewDefinition(SemanticAnalyzer.java:8781)
    at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:8689)
    at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:278)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:433)
    at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337)
    at org.apache.hadoop.hive.ql.Driver.run(Driver.java:902)
    at org.apache.hadoop.hive.service.HiveServer$HiveServerHandler.execute(HiveServer.java:198)
    at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)

Here is a working script:

CREATE TABLE Event1 (d STRING, userid BIGINT, eventData1 STRING);
CREATE TABLE Event2 (d STRING, userid BIGINT, eventData2 STRING);
CREATE VIEW AllEventsTest
  AS SELECT * FROM (
    SELECT 'Event1' AS eventType, map('d', `d`, 'userid', CAST(`userid` AS STRING)) AS eventData FROM Event1
    UNION ALL
    SELECT 'Event2' AS eventType, map('d', `d`, 'userid', CAST(`userid` AS STRING)) AS eventData FROM Event2
) d;

Issue Links

• is related to: HIVE-4547 A complex create view statement fails with new Antlr 3.4