Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Tasks involved:
1. Switch over from FM-sketch to HLL bit vectors to compute ndvs.
2. Store these bit vectors in RDBMS metastore. This code already exists for HBase metastore.
3. Combine bit vectors requested for partition list to get better ndv estimate. This can be done initially only for CachedStore to avoid implementation complexity.