Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16687

Native memory leak by Unsafe_allocatememory in Kafka Clients 3.7.0

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Invalid
    • 3.7.0
    • None
    • clients, consumer
    • None
    • Important

    Description

      I am building a Java Project which using Maven dependency Kafka-clients with 3.7.0 version.
      My Java application logic is to use Kafka Consumer to poll Kakfa broker topic  continuously. 
      I have configured my Java application with JVM options with -Xms8G -Xmx8G  -XX:MaxMetaspaceSize=4G, and then run it. 
      Also, there are 16G physical memory on my virtual machine. 
      After my Java application running a long time, I have found that resident memory of the Java Process was being grown to more than 14G.
      In the end, the Java process ate Swap space. 
      I checked it with jmap -heap pid, and found heap memory usage is Ok. 
      Also with Native Memory Tracking [jcmd pid Native.memory detail.diff], I found that it's caused by [NMT Internal] memory,  which created by Unsafe_allocatememory xxx.
      In my Java application, I don't use any NIO DirectByteBuffer to allocate memory.
      And I check it the Kafka-clients source code, it have codes with use "sun.misc.unsafe" to allocate memory.  And MaxMetaspaceSize not work for it .  
      Could you help to check it? How could I to stop this growing native memory to avoid my System hang?

      Attachments

        1. diff
          105 kB
          Philip Nee

        Activity

          People

            pnee Philip Nee
            fortherightous FTR
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: