[DRILL-4742] Using convert_from timestamp_impala gives a random error

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Cannot Reproduce
• Affects Version/s: 1.6.0, 1.7.0
• Fix Version/s: 1.14.0
• Assignee: Vitalii Diravka
• Reporter: Rahul Kumar Challapalli
• Created: 21/Jun/16 21:06
• Updated: 13/Jul/18 17:27
• Resolved: 12/Jul/18 15:55

Description

Drill Commit # fbdd20e54351879200184b478c2a32f238bf2176

The following query randomly generates the below error.

select convert_from(create_timestamp, 'TIMESTAMP_IMPALA') from dfs.`/drill/testdata/temp.parquet`;
Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: 0

Fragment 0:0

[Error Id: 9fe53a95-c4ae-424d-8c6d-489abab2d2ca on qa-node190.qa.lab:31010] (state=,code=0)

The underlying parquet file is generated using hive. Below is the metadata information

/root/parquet-tools-1.5.1-SNAPSHOT/parquet-meta temp.parquet 
creator:          parquet-mr version 1.6.0 

file schema:      hive_schema 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
voter_id:         OPTIONAL INT32 R:0 D:1
name:             OPTIONAL BINARY O:UTF8 R:0 D:1
age:              OPTIONAL INT32 R:0 D:1
registration:     OPTIONAL BINARY O:UTF8 R:0 D:1
contributions:    OPTIONAL FLOAT R:0 D:1
voterzone:        OPTIONAL INT32 R:0 D:1
create_timestamp: OPTIONAL INT96 R:0 D:1
create_date:      OPTIONAL INT32 O:DATE R:0 D:1

row group 1:      RC:200 TS:9902 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
voter_id:          INT32 UNCOMPRESSED DO:0 FPO:4 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
name:              BINARY UNCOMPRESSED DO:0 FPO:847 SZ:3214/3214/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
age:               INT32 UNCOMPRESSED DO:0 FPO:4061 SZ:438/438/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
registration:      BINARY UNCOMPRESSED DO:0 FPO:4499 SZ:241/241/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
contributions:     FLOAT UNCOMPRESSED DO:0 FPO:4740 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
voterzone:         INT32 UNCOMPRESSED DO:0 FPO:5583 SZ:843/843/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN
create_timestamp:  INT96 UNCOMPRESSED DO:0 FPO:6426 SZ:2642/2642/1.00 VC:200 ENC:PLAIN_DICTIONARY,RLE,BIT_PACKED
create_date:       INT32 UNCOMPRESSED DO:0 FPO:9068 SZ:838/838/1.00 VC:200 ENC:RLE,BIT_PACKED,PLAIN

I attached the log file and the data file

Issue Links

• is related to: DRILL-5495 convert_from function on top of int96 data results in ArrayIndexOutOfBoundsException
• relates to: DRILL-4337 Drill fails to read INT96 fields from hive generated parquet files