Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-4552

Optimize GridDhtLocalPartition.rmvQueue

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.6
    • 2.0
    • cache
    • None

    Description

      Current implementation stores deferred entry removals in rmvQueue for consistency guaranties.

      This can lead to significant heap over-usage(I observed several Gbs) in case of many caches with removals, because currently queue is cleared lazily after reaching max capacity(200_000 by default).

      This can be mitigated by using lower IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE, but can lead to consistency issues in case of frequent cache updates.

      Possible optimizations:

      • Use single fixed size queue per all caches to overcome limitations of IGNITE_ATOMIC_CACHE_DELETE_HISTORY_SIZE workaround.
      • Do queue cleaning in background
      • Move queue to an off-heap.

      Attachments

        1. Plot_ThroughputLatencyProbe_01_20170203_1.png
          61 kB
          Stanilovsky Evgeny
        2. Plot_ThroughputLatencyProbe_01_31_origin.png
          52 kB
          Evgeny Stanilovsky
        3. Plot_ThroughputLatencyProbe_01_4552_mine.png
          50 kB
          Evgeny Stanilovsky
        4. Plot_ThroughputLatencyProbe_01_mine.png
          59 kB
          Evgeny Stanilovsky
        5. Plot_ThroughputLatencyProbe_01_origin.png
          43 kB
          Evgeny Stanilovsky
        6. benchmark-put-remove-simultaneously.properties
          2 kB
          Evgeny Stanilovsky
        7. Plot_ThroughputLatencyProbe_01_origin_3node.png
          59 kB
          Evgeny Stanilovsky
        8. Plot_ThroughputLatencyProbe_01_mine_3node.png
          58 kB
          Evgeny Stanilovsky
        9. Screenshot_20170124_155355.png
          20 kB
          Evgeny Stanilovsky

        Activity

          People

            Unassigned Unassigned
            ascherbakov Alexey Scherbakov
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: