[CARBONDATA-3700] Optimize prune performance when prunning with multi-threads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: data-query
• Assignee: Unassigned
• Reporter: Xingjun Hao
• Created: 13/Feb/20 07:40
• Updated: 01/Apr/20 09:38
• Resolved: 01/Apr/20 09:38

Description

When pruning with multi-threads, there is a bug hambers the prunning performance heavily. 

When the datamap pruning results in no blocklet to map filter, The getExtendblocklet function aims to get the extend blocklet metadata, when the Input is a empty blocklet list, this function should return a extend blocklet list directyly , but now there is a bug leading to a hashset add operation overhead.

Meanwhile ,When pruning with multi-threads, the getExtendblocklet function will be triggerd for each blocklet. This should avoided by trgger this function for each segment.

 

Issue Links

• links to: GitHub Pull Request #3620