-
Type:
Improvement
-
Status: Closed
-
Priority:
Major
-
Resolution: Fixed
-
Affects Version/s: v1.3.0, v1.5.0
-
Fix Version/s: v1.5.3
-
Component/s: Job Engine
-
Labels:None
After KYLIN-1186, we've gained the ability to count distinct Int type columns precisely.
However, the implements of KYLIN-1186 is not stable, especially in 2.x-staging branch.
The reason is that the measure's maxlength is used to allocate memory in 2.x version, and the BitmapMeasure is hardcoded to 8MB in KYLIN-1186, causing OOM when cube building.
To resolve this problem, we have introduce precision on the bitmap measure, such as bitmap(100), bitmap(10000), bitmap(1000000), meaning the measure could accept 100/10000/1M cardinality at most. This solution should be fine, considering the reality, if the count value over 1000000, the hyperloglog measure which produce approx. result should be acceptable.
- depends upon
-
KYLIN-1705 Global (and more scalable) dictionary
-
- Closed
-
-
KYLIN-1718 Grow ByteBuffer Dynamically in Cube Building and Query
-
- Closed
-
-
KYLIN-1719 Add config in scan request to control compress the query result or not
-
- Closed
-
- relates to
-
KYLIN-1186 Support precise Count Distinct using bitmap (under limited conditions)
-
- Closed
-