[NIFI-5501] MySQL connection cleanup thread leak

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0-M1, 1.21.0
• Component/s: Core Framework
• Environment: - Ubuntu 18.04 LTS ; - mysql-connector-java-5.1.46 ; - Java 1.8.0_171
• Assignee: Matt Burgess
• Reporter: Tanapol Nearunchorn
• Created: 08/Aug/18 16:31
• Updated: 10/Mar/23 20:44
• Resolved: 10/Mar/23 20:38

Description

There are thousands of "Abandoned connection cleanup thread" left running that cause memory leak in NiFi.

I got a thread dump example here (full thread dump also attached):

"Abandoned connection cleanup thread" #18371 daemon prio=5 os_prio=0 tid=0x00007f3b840e7800 nid=0x76a3 in Object.wait() [0x00007f3b24ebb000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:143)
- locked <0x0000000348a12628> (a java.lang.ref.ReferenceQueue$Lock)
at com.mysql.jdbc.AbandonedConnectionCleanupThread.run(AbandonedConnectionCleanupThread.java:64)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Locked ownable synchronizers:
- <0x0000000348a12648> (a java.util.concurrent.ThreadPoolExecutor$Worker)

I'm not sure where these threads come from which component because in my flow, I used CaptureChangeMySQL processor and DBCPConnectionPool controller service.

As I also found related problem here: mysql Bug #69526 but it quite a bit old.

Issue Links

• links to: GitHub Pull Request #7031