Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-8892

Iterating over large dataset via ScanQuery can fails with OOME.

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 2.7
    • cache

    Description

      Seems, iterating over query iterator (ScanQuery at least, but may be other affected as well) on client node cause memory leakage.

      The use case is quite simple.
      Start server and client. Put much data into cache, then iterate over all entries via ScanQuery.
      Looks like JVM crashed due to OOM as GridCacheDistributedQueryFuture.allCol map contains to many entries.

      I've put 15kk entries into cache and client failed with OOM after iterating over 10kk entry.
      In heapdump I observer 10kk GridCacheDistributedQueryFuture entries. 

      We have to check if collection cleared correctly and it is really need to collect all entries.

      PFA repro.

      Attachments

        1. ScanQueryOOM.java
          4 kB
          Andrey Mashenkov
        2. list-8892.jfr
          1.47 MB
          Evgeny Stanilovsky
        3. list-master.jfr
          1.30 MB
          Evgeny Stanilovsky

        Activity

          People

            amashenkov Andrey Mashenkov
            amashenkov Andrey Mashenkov
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 10m
                10m