[CARBONDATA-1542] Add support of Fine Grained (FG) datamap to get the pruning done in rowid level.

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Ravindra Pesala
• Created: 09/Oct/17 08:43
• Updated: 09/Oct/17 08:43

Description

Currently, CarbonData supports Coarse-grained (CG) datamap, it can prune the data only till blocklet level. So its pruning is limited till driver level for the first level of pruning.But there are scenarios like supporting Lucene index requires pruning till row level so Fine Grained (FG) datamap is required to support these kinds of features.
So please support FG dataMap.

Sub-Tasks

• Add interfaces for FG datamap: Open
• Implement DataMapChooser to chose suitable datamap depends on the filter condition.: Open
• Implement bitset pipe-lining in carbondata filtering. It will improve performance and needed for FG Datamap: Resolved
• Change FilterScanner to do FG datamap prune first and then allow for further filter scan: Open
• Integrate and update datamap as per change events of alter and update.: Open