Details
-
Sub-task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
Implementing distributed cube materialization on holistic measure based on MR-Cube approach as described in http://arnab.org/files/mrcube.pdf.
Primary steps involved:
1) Identify if the measure is holistic or not
2) Determine algebraic attribute (can be detected automatically for few cases, if automatic detection fails user should hint the algebraic attribute)
3) Modify MRPlan to insert a sampling job which executes naive cube algorithm and generates annotated cube lattice (contains large group partitioning information)
4) Modify plan to distribute annotated cube lattice to all mappers using distributed cache
5) Execute actual cube materialization on full dataset
6) Modify MRPlan to insert a post process job for combining the results of actual cube materialization job
7) OOM exception handling