[FLINK-26593] task manager can not free memory when jobs are finished caused the linux out of memory

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.14.3
• Component/s: API / Core

Environment

The flink cluster environment with five nodes

The iceberg is 0.13.1 for flink 1.14

The hadoop is 3.3.1

The hive is 3.1.2

 

 

• Assignee: Unassigned
• Reporter: ren shangtao
• Created: 11/Mar/22 02:22
• Updated: 11/Mar/22 08:37

Description

I have A batch task that inserts 100 million data from table A into table B after the primary key hash. I use Flink standalone mode to execute this task. After several times of executing this task, I find that the Flink TaskManager process occupies more and more memory. Finally, the Flink TaskManager process is killed when the system runs out of memory.

 

Here is the message

 
Before run the job

 
After serval times 

 
My configuration