[CARBONDATA-3436] update pre insert into rule as per spark

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Assignee: Unassigned
• Reporter: Ajantha Bhat
• Created: 13/Jun/19 14:29
• Updated: 18/Jun/19 05:15
• Resolved: 18/Jun/19 05:15

Description

Problem: Carbon is not following hive syntax for one insert into strictly.

Cause: For insert into, carbon has its own rule. CarbonAnalysisRule[CarbonPreInsertionCasts]
In CarbonPreInsertionCasts, data type cast is missed. Hence carbon was not following hive syntax.

solution:
Add data type cast rule for insert into.

Note:

1. Change syntax for insert single record(for binary and complex type(array,struct,map))
2. Binary cannot accept value as integer, need to pass in quotes only.
3. Earlier complex type was accepting null values for complex datatypes, now it won't accept null values for complex types as per Hive rule.
4. Need to pass in default format only for date type and timestamp type for Complex type in insert command.
5. For boolean type 't' is a valid entry for true and 'f' is a valid entry for false.
6. For boolean type , any integer greater than 0 will be considered as true and 0 will be considered as false.
7. In case of map,for same key. As per hive, the latter value should be inserted for the same key.

Steps to reproduce:

create table test(name string,id binary) stored by 'carbondata';
insert into test select 'a',1;

insert should throw cast exception as 1 is not binary. But it is passing now.

 

Issue Links

• links to: GitHub Pull Request #3244