[KYLIN-4188] Parquet as Cube storage V2

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Fix Version/s: v4.0.0-alpha
• Component/s: Job Engine, Query Engine, Storage - Parquet
• Assignee: Shao Feng Shi
• Reporter: nichunen
• Created: 08/Oct/19 03:55
• Updated: 13/Sep/20 12:16
• Resolved: 03/Sep/20 09:34

Description

Branch "kylin-on-parquet" has implemented and it's ready for public review and evaluation, the issue is https://issues.apache.org/jira/browse/KYLIN-3621.

However, as the discussion in https://sematext.com/opensee/m/Kylin/8WImho86LC1RxadI?subj=Evaluate+Kylin+on+Parquet, to make it more stable and with higher performance, some tasks of improvement should be done.

The key tasks may be:
1. Calculation of queries on Kylin should be all distributed
2. Remove dictionaries for dimensions
3. Store value with its data type(not binary)
4. The new build engine with Spark-SQL

Issue Links

• Blocked: KYLIN-4214 Distributed query engine
• Blocked: KYLIN-4213 The new build engine with Spark-SQL
• is related to: KYLIN-4659 Prepare a technical preview version for Parquet Storage