Details
-
Improvement
-
Status: Open
-
P3
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Requires BEAM-4276 to be completed first.
Additional performance tweaks to the in memory per-key accumulator cache during the pre-combine phase of a lifted combine.
This can include any of
- capping the number of key-accumulator in the cache, and draining them eagerly after x elements seen, or evicting keys by some heuristic after the cache has seen X distinct keys.
- providing a counter on cache size (key/element counts) exposable through the metrics interface or another runner standard counter, to permit observing the cache's status, especially if it could grow without bound within a bundle.
Attachments
Issue Links
- links to