[CARBONDATA-3306] Implement a DistributableIndexPruneRDD and IndexPruneFileFormat

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.2
• Fix Version/s: 1.6.0
• Flags: Patch

Docs Text

The design document is in the https://docs.google.com/document/d/161NXxrKLPucIExkWip5mX00x2iOPH6bvsuQnCzzp47E/edit?ts=5c542ab4#heading=h.x0qaehgkncz5

• Assignee: Unassigned
• Reporter: TaoLi
• Created: 04/Mar/19 07:11
• Updated: 20/May/19 05:48
• Resolved: 20/May/19 05:48

Description

○ The RDD will accept list of segments and the filter expression for pruning.

○ The pruning job is handled by the executors in a distributed way for the allocated segments.

○ The splits are equally divided based on the size on the index files.

Issue Links

• links to: GitHub Pull Request #3177