[IMPALA-723] Union queries give Wrong result in a UNION followed by SIGSEGV in another union

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 1.2.3
• Fix Version/s: Impala 1.2.4, Impala 1.3
• Labels: impala
• Environment: CDH 4.5 / Impala 1.2.3 / Centos 6
• Assignee: Unassigned
• Reporter: Manoj Samel
• Created: 27/Dec/13 20:07
• Updated: 20/Dec/15 00:05
• Resolved: 08/Jan/14 22:51

Description

Create a simple table in impala parquet format

use impala;
create table currency_rate (
rate_type STRING,
effective_date TIMESTAMP,
from_currency STRING,
to_currency STRING,
conversion_factor double
)
STORED AS PARQUETFILE
;

Loaded some data from a hive CSV external table using insert into impala.currency_rate select * from manoj.currency_rate ... (note hive use string instead of time stamp)

use manoj;
create external table currency_rate (
rate_type STRING,
effective_date STRING,
from_currency STRING,
to_currency STRING,
conversion_factor double
)
row format delimited fields terminated by ','
STORED AS TEXTFILE
LOCATION '/foo/currency_rate'
;

The CSV data file is

fixed,1900-01-01,1698$7,1698$34,1.0
fixed,1900-01-01,1698$34,1698$7,1.0
fixed,2000-01-01,1698$7,1698$34,2.0
fixed,2000-01-01,1698$34,1698$7,0.5
fixed,2009-01-01,1698$7,1698$1,2.0
fixed,2009-01-01,1698$1,1698$7,0.5
fixed,2010-01-01,1698$7,1698$34,4.0
fixed,2010-01-01,1698$34,1698$7,0.25
fixed,2014-01-01,1698$7,1698$1,4.0
fixed,2014-01-01,1698$1,1698$7,0.25
fixed,2015-01-01,1698$7,1698$34,8.0
fixed,2015-01-01,1698$34,1698$7,0.125

Using impala-shell

0) use impala;

1) select rate_type, sum(conversion_factor) from currency_rate group by rate_type; gives correct results 1 row since there is only one rate_type

fixed | 23.625

2) select rate_type, sum(conversion_factor) from currency_rate group by rate_type union select rate_type, 1 from currency_rate

Gives first row that is malformed - instead of showing ("fixed", 23.635) as in #1, it shows "", 47.25

47.25
fixed	1

3) Query: select rate_type, sum(conversion_factor) from currency_rate group by rate_type union select rate_type, sum(conversion_factor) from currency_rate group by rate_type (problem)
ERROR: Couldn't open transport for ip-10-222-0-143:22000(connect() failed: Connection refused)

4) Query: select rate_type, sum(conversion_factor) from currency_rate group by rate_type (Still alive)
---------------------------------+

rate_type

sum(conversion_factor)

---------------------------------+

fixed

23.625

5) select rate_type, sum(conversion_factor) from currency_rate group by rate_type union select rate_type, conversion_factor from currency_rate; (Dies here !)
Query: select rate_type, sum(conversion_factor) from currency_rate group by rate_type union select rate_type, conversion_factor from currency_rate
Error communicating with impalad: TSocket read 0 bytes

impala-server.log says

#

1. A fatal error has been detected by the Java Runtime Environment:
   #
2. SIGSEGV (0xb) at pc=0x0000000000a18b12, pid=15528, tid=140059349042944
   #
3. JRE version: 6.0_45-b06
4. Java VM: Java HotSpot(TM) 64-Bit Server VM (20.45-b01 mixed mode linux-amd64 compressed oops)
5. Problematic frame:
6. C [impalad+0x618b12] impala::AggregationNode::Prepare(impala::RuntimeState*)+0x782
   #
7. An error report file with more information is saved as:
8. /var/run/impala/hs_err_pid15528.log
   #
9. If you would like to submit a bug report, please visit:
10. http://java.sun.com/webapps/bugreport/crash.jsp
11. The crash happened outside the Java Virtual Machine in native code.
12. See problematic frame for where to report the bug.
    #

• • • See attached dumped file ***