Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-8006

Starting multiple caches slows down exchange process on joining node

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8
    • Component/s: None
    • Labels:
      None

      Description

      In some cases when we starts multiple caches (over 2K caches), we can get a stop on exchange when new node joining to the cluster.

      Coordinator-node wait to receive a single message from all other nodes, but last node (which want to joining to the cluster) stopped on starting caches:

       

      Stack trace
       at java.lang.Thread.dumpStack(Thread.java:1329)
       at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCache(GridCacheProcessor.java:1159)
       at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1900)
       at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1764)
       at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:740)
       at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:622)
       at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2329)
       at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
       at java.lang.Thread.run(Thread.java:745)
      
      

      It blocks cluster exchange process until all caches started on the last node.

      We should start caches in parallel threads or exclude the action from exchange init process.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                akalashnikov Anton Kalashnikov
                Reporter:
                v.pyatkov Vladislav Pyatkov
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: