Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
v1.6.0
-
None
-
Patch
Description
Kylin's aggregation operator can spill intermediate results to disk when its estimated memory usage exceeds some threshold (kylin.query.coprocessor.mem.gb to be specific). While it's a useful feature in general to prevent RegionServer from OOM, there are times when aborting this kind of memory-hungry query immediately is a more suitable choice to users.
To accommodate this requirement, I suggest adding a new configuration named kylin.storage.hbase.coprocessor-spill-enabled kylin.storage.partition.aggr-spill-enabled. The default value would be true, which will keep the same behavior as before. If changed to false, query that uses more aggregation memory than threshold will fail immediately.