Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-3018

Cache affinity calculation is slow with large nodes number

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.0
    • cache

    Description

      With large number of cache server nodes (> 200) RendezvousAffinityFunction and FairAffinityFunction work pretty slow .

      For RendezvousAffinityFunction.assignPartitions can take hundredes of milliseconds, for FairAffinityFunction it can take seconds.

      For RendezvousAffinityFunction most time is spent in MD5 hash calculation and nodes list sorting. As optimization we can try to cache

      {partion, node}

      MD5 hash or try another hash function. Also several minor optimizations are possible (avoid unncecessary allocations, only one thread local 'get', etc).

      Attachments

        1. 600.png
          16 kB
          Taras Ledkov
        2. 400.png
          14 kB
          Taras Ledkov
        3. 256.png
          14 kB
          Taras Ledkov
        4. 200.png
          14 kB
          Taras Ledkov
        5. 128.png
          13 kB
          Taras Ledkov
        6. 100.png
          13 kB
          Taras Ledkov
        7. 064.png
          12 kB
          Taras Ledkov
        8. 016.png
          23 kB
          Taras Ledkov
        9. 008.png
          19 kB
          Taras Ledkov
        10. 004.png
          16 kB
          Taras Ledkov
        11. 003.png
          14 kB
          Taras Ledkov
        12. balanced.600.png
          15 kB
          Taras Ledkov
        13. balanced.400.png
          14 kB
          Taras Ledkov
        14. balanced.256.png
          14 kB
          Taras Ledkov
        15. balanced.200.png
          13 kB
          Taras Ledkov
        16. balanced.128.png
          13 kB
          Taras Ledkov
        17. balanced.100.png
          13 kB
          Taras Ledkov
        18. balanced.064.png
          12 kB
          Taras Ledkov
        19. balanced.016.png
          23 kB
          Taras Ledkov
        20. balanced.008.png
          19 kB
          Taras Ledkov
        21. balanced.004.png
          17 kB
          Taras Ledkov
        22. balanced.003.png
          14 kB
          Taras Ledkov

        Issue Links

          causes

          Bug - A problem which impairs or prevents the functions of the product. IGNITE-12896 excludeNeighbors=true breaks zone-awareness

          • Major - Major loss of function.
          • Open
          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. IGNITE-3905 Optimize RendezvousAffinityFunction

          • Critical - Crashes, loss of data, severe memory leak.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. IGNITE-4765 Different partition mapping for caches that use Fair Affinity Function

          • Major - Major loss of function.
          • Closed
          relates to

          Improvement - An improvement or enhancement to an existing feature or task. IGNITE-4756 Print info about partition distribution to log

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          links to

          Web Link GitHub Pull Request #684

          Web Link GitHub Pull Request #1575

          Web Link GitHub Pull Request #1647

          Web Link GitHub Pull Request #1779

          Activity

            People

              tledkov-gridgain Taras Ledkov
              sboikov Semen Boikov
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: