Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-2389

Need to add ability to swap the queue if it's too long

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: data structures
    • Labels:
      None

      Description

      In some cases there can be a requirement to start swapping new queue elements to avoid out of memory errors. For example, if the consumer temporarily stopped for some reason, but the producer still adds elements.

      We can do this based on cache swap space and evictions. I think we should:

      1. Add max allowed in-memory elements property to CollectionConfiguration. We can also support limit by memory size.
      2. Implement special eviction policy that will evict new queue elements if limits are reached.
      3. Automatically unswap latest elements when consumer resumes polling (can be achieved by a background process).

      In case of IgniteSet, limits should be also honored, but here we don't have ordering. So I think we can use our standard LRU, but make sure that only elements are swapped. Header and other meta objects should be always in memory.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              vkulichenko Valentin Kulichenko
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: