Resolution: Won't Fix
Affects Version/s: None
Fix Version/s: None
Component/s: data structures
In some cases there can be a requirement to start swapping new queue elements to avoid out of memory errors. For example, if the consumer temporarily stopped for some reason, but the producer still adds elements.
We can do this based on cache swap space and evictions. I think we should:
- Add max allowed in-memory elements property to CollectionConfiguration. We can also support limit by memory size.
- Implement special eviction policy that will evict new queue elements if limits are reached.
- Automatically unswap latest elements when consumer resumes polling (can be achieved by a background process).
In case of IgniteSet, limits should be also honored, but here we don't have ordering. So I think we can use our standard LRU, but make sure that only elements are swapped. Header and other meta objects should be always in memory.