[CARBONDATA-2605] Complex DataType Enhancements

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: dhatchayani
• Reporter: sounak chakraborty
• Created: 13/Jun/18 14:20
• Updated: 10/Dec/18 13:58

Description

Umbrella Jira to implement enhancements in Complex Data Type for Carbon.

• Projection push down for struct data type.
• Provide adaptive encoding and decoding for all data type.
• Support JSON data loading directly into Carbon table.

 

Please access the Design Document through this link. 

 

https://docs.google.com/document/d/12EZwUlLs53Vro7pMeLnFd0lCjeKOakKY-60e3cryJb4/edit#

Issue Links

• is a parent of: CARBONDATA-2798 Fix Dictionary_Include for ComplexDataType
• is related to: CARBONDATA-2395 Not able to fetch all data from a complex data type table
• is related to: CARBONDATA-2397 Error while fetching data from a table with complex data type(array_of_struct)
• is related to: CARBONDATA-2398 Getting Error while executing concat function on complex data type.
• is related to: CARBONDATA-2399 Getting Error while applying filter on complex data type
• is related to: CARBONDATA-1457 Stabilize Struct DataType Support
• is related to: CARBONDATA-1489 Stabilize Array DataType Support

Sub-Tasks

• Projection push down for struct data type: Resolved
• Provide Adaptive Encoding and Decoding for all data type: Resolved
• SDK Support JSON data loading directly without AVRO conversion: Resolved
• Complex DatatYpe Enhancements. Pending Comments of CARBONDATA-2607: Open
• [BUG] support struct of date in create table: Resolved
• Compaction of Complex DataType (STRUCT AND ARRAY): Resolved
• Avoid duplicate decoding for complex column pages while querying: Resolved
• Compaction support with MAP data type: Resolved