[CARBONDATA-2804] Incorrect error message when bloom filter or preaggregate datamap tried to be created on older V1-V2 version stores

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.4.1
• Fix Version/s: 1.4.1
• Component/s: data-query
• Environment: Spark 2.1
• Assignee: wangsen
• Reporter: Chetan Bhat
• Due: 03/Aug/18
• Created: 30/Jul/18 14:22
• Updated: 03/Aug/18 16:28
• Resolved: 03/Aug/18 07:27

Description

Steps :

User creates a table with V1 version store and loads data to the table.

create table brinjal (imei string,AMSize string,channelsId string,ActiveCountry string, Activecity string,gamePointId double,deviceInformationId double,productionDate Timestamp,deliveryDate timestamp,deliverycharge double) STORED BY 'org.apache.carbondata.format' TBLPROPERTIES('table_blocksize'='1');
LOAD DATA INPATH 'hdfs://hacluster/chetan/vardhandaterestruct.csv' INTO TABLE brinjal OPTIONS('DELIMITER'=',', 'QUOTECHAR'= '"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'= 'imei,deviceInformationId,AMSize,channelsId,ActiveCountry,Activecity,gamePointId,productionDate,deliveryDate,deliverycharge');

In 1.4.1 version user refreshes the table with V1 store and tries to create a bloom filter datamap.

CREATE DATAMAP dm_brinjal ON TABLE brinjal2 USING 'bloomfilter' DMPROPERTIES ('INDEX_COLUMNS' = 'AMSize', 'BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001');

create datamap brinjal_agg on table brinjal2 using 'preaggregate' as select AMSize, avg(gamePointId) from brinjal group by gamePointId, AMSize;

Issue : Bloom filter or preaggregate datamap fails with incorrect error message.

0: jdbc:hive2://10.18.98.101:22550/default> CREATE DATAMAP dm_brinjal ON TABLE brinjal2 USING 'bloomfilter' DMPROPERTIES ('INDEX_COLUMNS' = 'AMSize', 'BLOOM_SIZE'='640000', 'BLOOM_FPP'='0.00001');
Error: java.io.IOException: org.apache.thrift.protocol.TProtocolException: Required field 'version' was not found in serialized data! Struct: org.apache.carbondata.format.FileHeader$FileHeaderStandardScheme@4d5aa8b2 (state=,code=0)

 0: jdbc:hive2://10.18.98.101:22550/default> create datamap brinjal_agg on table brinjal2 using 'preaggregate' as select AMSize, avg(gamePointId) from brinjal group by gamePointId, AMSize;
Error: java.io.IOException: org.apache.thrift.protocol.TProtocolException: Required field 'version' was not found in serialized data! Struct: org.apache.carbondata.format.FileHeader$FileHeaderStandardScheme@55d8323c (state=,code=0)

Expected : Correct error message should be displayed when bloom filter or preaggregate datamap creation is blocked/fails.

Issue Links

• links to: GitHub Pull Request #2601