[CARBONDATA-1457] Stabilize Struct DataType Support

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.1.1
• Component/s: core, sql
• Assignee: Venkata Gollamudi
• Reporter: Sharanabasappa G Keriwaddi
• Due: 20/Sep/17
• Created: 06/Sep/17 12:12
• Updated: 11/Jul/18 06:19

Description

Stabilize Struct DataType Support. This is umbrella jira to track all related tasks.
Following cases needs to be handled:

Sub feature	Pending activity	Remarks
Load,query, filter, NULL values, UDFs, Describe support	Test + Fix	Implementation in place needs to add test-cases and bug fix
Compaction support	Test + fix	As compaction works at byte level, no changes required. Needs to add test-cases
Insert into table	Develop	Source table data containing struct data needs to convert from spark datatype to string , as carbon takes string as input row
Support DDL for Struct fields Dictionary include and Dictionary Exclude	Develop	Also needs to handle CarbonDictionaryDecoder to handle the same.
Support multilevel struct	Develop	currently DDL is validated to allow only 2 levels, remove this restriction
Support struct field to be a measure	Develop	Currently supports only dimensions
Support Alter table to add and remove struct column	Develop	implement DDL and requires default value handling
Update & Delete Support	Test + Fix	implementation is in place. Need to enhance test-cases and bug fix
Projections of struct fields push down to carbon	Develop	this is an optimization, when more number of fields are present in struct
Filter struct fields push down to carbon	Develop	this is an optimization, when more number of fields are present in struct
Struct fields participate in sort column	Develop	This is can be low priority

Issue Links

• relates to: CARBONDATA-1487 problem with the conversion data , using the carbondata struct type
• relates to: CARBONDATA-1143 Incorrect Data load while loading data into struct of struct
• relates to: CARBONDATA-1360 Update is not working properly for complex datatype
• relates to: CARBONDATA-1489 Stabilize Array DataType Support
• relates to: CARBONDATA-2605 Complex DataType Enhancements

Sub-Tasks

• Alter add and remove struct Column: Open
• Compaction Support: Enhance Test Cases for Struct DataType: Open
• Load,query, filter, NULL values, UDFs, Describe support: Open
• Projections of struct fields push down to carbon: In Progress