[BAHIR-324] JobManager created a kuduReader that was not closed, resulting in an increase in server handles

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Flink-1.1.0
• Fix Version/s: Flink-1.2.0
• Component/s: Flink Streaming Connectors
• Flags: Patch

Language

JAVA

• Assignee: ShiminHuang
• Reporter: ShiminHuang
• Created: 30/May/23 07:07
• Updated: 30/May/23 16:37
• Resolved: 30/May/23 08:27

Description

Problem

When JobManager starts, the ExecutionJobVertex constructor calls the AbstractKuduInputFormat#createInputSplits method, Causes the KuduReader object to create the KuduClient and KuduSession objects, but ExecutionJobVertex does not call the Kudureader #close() method with the display, causing the Kudu-related handle to remain open after the Flink task is closed. As a result, the number of server handles increases with the number of Flink tasks. As a result, the entire server becomes unavailable

Code

ExecutionJobVertex#constructor

AbstractKuduInputFormat#createInputSplits{}

AbstractKuduInputFormat#startKuduReader

{}

Issue Links

• links to: GitHub Pull Request #166