[FLINK-19005] Document JDBC drivers as source of Metaspace leaks

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.1
• Fix Version/s: 1.12.0
• Component/s: Client / Job Submission, Runtime / Configuration, Runtime / Coordination
• Labels: pull-request-available
• Assignee: Chesnay Schepler
• Reporter: Guillermo Sánchez
• Created: 20/Aug/20 12:24
• Updated: 17/Sep/20 10:28
• Resolved: 17/Sep/20 09:08

Description

Hi !

Im running a 1.11.1 flink cluster, where I execute batch jobs made with DataSet API.

I submit these jobs every day to calculate daily data.

In every execution, cluster's used metaspace increase by 7MB and its never released.

This ends up with an OutOfMemoryError caused by Metaspace every 15 days and i need to restart the cluster to clean the metaspace

taskmanager.memory.jvm-metaspace.size is set to 512mb

Any idea of what could be causing this metaspace grow and why is it not released ?

 

================================================
=== Summary ======================================
================================================

Case 1, reported by gestevez:

• Flink 1.11.1
• Java 11
• Maximum Metaspace size set to 512mb
• Custom Batch job, submitted daily
• Requires restart every 15 days after an OOM

 Case 2, reported by Echo Lee:

• Flink 1.11.0
• Java 11
• G1GC
• WordCount Batch job, submitted every second / every 5 minutes
• eventually fails TaskExecutor with OOM

Case 3, reported by DaDaShen

• Flink 1.11.0
• Java 11
• WordCount Batch job, submitted every 5 seconds
• growing Metaspace, eventually OOM
   

Issue Links

• is related to: FLINK-16408 Bind user code class loader to lifetime of a slot
• links to: GitHub Pull Request #13395