[CARBONDATA-1516] Support pre-aggregate tables and timeseries in carbondata

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.0
• Assignee: Kumar Vishal
• Reporter: Ravindra Pesala
• Created: 27/Sep/17 09:47
• Updated: 16/Oct/18 09:30
• Resolved: 29/Mar/18 13:01

Description

Currently Carbondata has standard SQL capability on distributed data sets.Carbondata should support pre-aggregating tables for timeseries and improve query performance.

Issue Links

• is a parent of: CARBONDATA-2088 Optimize syntax for creating timeseries pre-aggregate table

Sub-Tasks

• 1. Support CTAS in carbon and support creating aggregation tables using CTAS.And update aggregation table information to main table schema.: Resolved
• 2. Support creating timeseries while creating main table.: Resolved
• 3. Create UDF for timestamp to extract year,month,day,hour and minute from timestamp and date: Resolved
• 4 Load aggregation tables from main table after finish.: Resolved
• 5. Add sql syntax for loading aggregation tables offline.: Open
• 6. Loading aggregation tables for streaming data tables.: Resolved
• . Add the API in carbon layer to get suitable aggregation table for group by query. Update query plan in carbon optimizer to support aggregation tables for group by queries.: Resolved
• 8. Refresh the cache of main table after droping of aggregation table.: Resolved
• 9. Drop all related aggregation tables upon deleting of main table.: Open
• 10. Handle compaction in aggregation tables.: Resolved
• 11. Don't allow update/delete and delete segment operations on main table which has aggregation tables.: Open
• 12. Handle alter table scenarios for aggregation table: Resolved
• 13. support describe for aggregation table: Open
• Support Desc table and desc formatted table for Pre Aggregate table: Open
• Update thrift to support Pre Aggregate support: Open
• Support expression inside aggregate expression in create and load data on Pre aggregate table: Resolved
• Support expression inside aggregate expression during query on Pre Aggregate table: Resolved
• Support sub query on Pre Aggregate table: Resolved
• Support Query on Pre Aggregate on streaming table: Open
• Support preaggregate table creation on partition tables: Resolved