[CARBONDATA-3464] TaskResultLost issue when executing select filter query

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.6.0
• Component/s: data-query
• Environment: Spark 2.1
• Assignee: Unassigned
• Reporter: Chetan Bhat
• Created: 04/Jul/19 06:24
• Updated: 04/Jul/19 06:24

Description

Steps :

From spark beeline Create table , load data multiple times and perform major compaction.

create table Array_com (CUST_ID string, YEAR int, MONTH int, AGE int, GENDER string, EDUCATED string, IS_MARRIED string, ARRAY_INT array<int>,ARRAY_STRING array<string>,ARRAY_DATE array<string>,CARD_COUNT int,DEBIT_COUNT int, CREDIT_COUNT int, DEPOSIT double, HQ_DEPOSIT double) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('DICTIONARY_INCLUDE'='ARRAY_INT,ARRAY_STRING,ARRAY_DATE,DEPOSIT,HQ_DEPOSIT');
LOAD DATA INPATH 'hdfs://hacluster/chetan/Array.csv' INTO table Array_com options ('DELIMITER'=',', 'QUOTECHAR'='"', 'FILEHEADER'='CUST_ID,YEAR,MONTH,AGE,GENDER,EDUCATED,IS_MARRIED,ARRAY_INT,ARRAY_STRING,ARRAY_DATE,CARD_COUNT,DEBIT_COUNT,CREDIT_COUNT,DEPOSIT,HQ_DEPOSIT','COMPLEX_DELIMITER_LEVEL_1'='$');
LOAD DATA INPATH 'hdfs://hacluster/chetan/Array.csv' INTO table Array_com options ('DELIMITER'=',', 'QUOTECHAR'='"', 'FILEHEADER'='CUST_ID,YEAR,MONTH,AGE,GENDER,EDUCATED,IS_MARRIED,ARRAY_INT,ARRAY_STRING,ARRAY_DATE,CARD_COUNT,DEBIT_COUNT,CREDIT_COUNT,DEPOSIT,HQ_DEPOSIT','COMPLEX_DELIMITER_LEVEL_1'='$');
LOAD DATA INPATH 'hdfs://hacluster/chetan/Array.csv' INTO table Array_com options ('DELIMITER'=',', 'QUOTECHAR'='"', 'FILEHEADER'='CUST_ID,YEAR,MONTH,AGE,GENDER,EDUCATED,IS_MARRIED,ARRAY_INT,ARRAY_STRING,ARRAY_DATE,CARD_COUNT,DEBIT_COUNT,CREDIT_COUNT,DEPOSIT,HQ_DEPOSIT','COMPLEX_DELIMITER_LEVEL_1'='$');
alter table Array_com compact 'major';

 

Execute select filter query

0: jdbc:hive2://10.18.98.120:22550/default> select array_string[1] from array_com where array_string[1] in (select array_string[1] from array_com) order by array_string[1];

Issue : TaskResultLost issue when executing select filter query

0: jdbc:hive2://10.18.98.120:22550/default> select array_string[1] from array_com where array_string[1] in (select array_string[1] from array_com) order by array_string[1];
Error: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 21074.0 failed 4 times, most recent failure: Lost task 0.3 in stage 21074.0 (TID 898569, BLR1000037944, executor 169): TaskResultLost (result lost from block manager)

Expected : Select filter query to be executed successfully.