[CARBONDATA-2632] BloomFilter DataMap Bugs and Optimization

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.1
• Assignee: Chuanyin Xu
• Reporter: Chuanyin Xu
• Created: 24/Jun/18 02:21
• Updated: 14/Aug/18 12:08
• Resolved: 14/Aug/18 12:08

Description

This is an umbrella Jira for bloomfilter bugs

Issue Links

• links to: GitHub Pull Request #2403
• links to: GitHub Pull Request #2408

Sub-Tasks

• Bugs are found when bloomindex column is dictionary/sort/date column: Resolved
• Provide more information about the datamap when showing datamaps: Resolved
• Support different provider based index datamaps on same column: Resolved
• Fix bugs for deferred rebuild for bloomfilter datamap: Resolved
• optimize blocklet pruning for bloomfilter: Open
• explain query shows negative skipped blocklets for bloomfilter datamap: Resolved
• Pipelined blocklet pruning for index datamaps: Open
• Fix bugs in incorrect blocklet number in bloomfilter: Resolved
• Optimize output for explaining query with datamap: Resolved
• Support `in` operator for bloomfilter datamap: Resolved
• Loading/Filtering empty value fails on bloom index columns: Resolved
• Support filtering on longstring bloom index columns: Resolved
• make datamap rebuild for all segments in parallel: Resolved
• update document for bloomfilter: Resolved
• Fix bug for alter rename is renameing the existing table on which bloomfilter datamp exists: Resolved
• Block alter datatype of bloom index columns: Resolved
• Whether to support altering name for bloom index columns: Resolved
• Block dropping index columns for index datamap: Resolved
• Fix bugs in clear bloom datamap: Resolved
• clear bloom index file after segment is deleted: Resolved
• clear index file if dataloading is failed: Resolved
• Refactor and move ReadSupport in IndexDataMapRebuildRDD to CarbonCore: Open
• Add validate for datamap writer while loading data: Resolved
• Table update/delete is needed block on table having datamaps: Resolved
• Failed to recreate the table which has bloomfilter on it with same table name but different bloom index: Resolved
• Support create bloom datamap on newly added column: Resolved
• Block create bloomfilter datamap index on local_dictionary column: Closed
• Block create bloomfilter datamap index on column which its datatype is complex type: Resolved
• Fix bug for getting datamap file when table has multiple datamaps: Resolved
• Fix bug when building bloomfilter on measure column: Resolved
• Optimize code to get blocklet id when rebuilding datamap: Resolved
• Exception should be thrown if expression do not satisfy bloomFilter's requirement: Resolved
• Update document of bloom filter datamap: Resolved
• Optimize default parameter for bloomfilter datamap: Resolved
• Fix bugs in incorrect query result with bloom datamap: Resolved
• Add useful tips for bloomfilter datamap: Resolved
• Add query test case using search mode on table with bloom filter: Resolved
• Merge bloom index files of multi-shards for each index column: Resolved
• Fix bug in bloom index on multiple dictionary columns: Resolved
• add sdv test case for bloomfilter datamap: Resolved