[FLINK-16710] Log Upload blocks Main Thread in TaskExecutor

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.11.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: jrthe42
• Reporter: Gary Yao
• Created: 22/Mar/20 12:22
• Updated: 03/Apr/20 19:38
• Resolved: 03/Apr/20 19:38

Description

Uploading logs to the BlobServer blocks the TaskExecutor's main thread. We should introduce an IO thread pool that carries out file system accesses (listing files in a directory, checking if file exists, uploading files).

Affected RPCs:

• TaskExecutor#requestLogList(Time)
• TaskExecutor#requestFileUploadByName(String, Time)
• TaskExecutor#requestFileUploadByType(FileType, Time)

Issue Links

• relates to: FLINK-16302 Add Rest Handler to list TM Logfiles and enable reading Logs by Filename
• links to: GitHub Pull Request #11571