[CARBONDATA-284] Abstracting Index and Segment interface

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.1.0-incubating
• Fix Version/s: 1.0.0-incubating
• Component/s: hadoop-integration
• Assignee: Jacky Li
• Reporter: Jacky Li
• Created: 03/Oct/16 16:12
• Updated: 05/Nov/16 00:38
• Resolved: 05/Nov/16 00:38

Description

This issue is intended to abstract developer API and user API to achieve following goals:

Goal 1: User can choose the place to store Index data, it can be stored in
processing framework's memory space (like in spark driver memory) or in
another service outside of the processing framework (like using a
independent database service, which can be shared across client)

Goal 2: Developer can add more index of his choice to CarbonData files.
Besides B+ tree on multi-dimensional key which current CarbonData supports,
developers are free to add other indexing technology to make certain
workload faster. These new indices should be added in a pluggable way.

This Jira has been discussed in maillist:
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Abstracting-CarbonData-s-Index-Interface-td1587.html

Issue Links

• links to: GitHub Pull Request #208
• links to: GitHub Pull Request #297