[FLINK-18097] History server doesn't clean all job json files

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.10.1
• Fix Version/s: 1.10.2, 1.11.1, 1.12.0
• Component/s: Runtime / REST
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Milan Nikl
• Created: 03/Jun/20 20:07
• Updated: 26/Jun/21 20:23
• Resolved: 08/Jul/20 19:11

Description

Improvement introduced in https://issues.apache.org/jira/browse/FLINK-14169 does not delete all files in the history server folders completely.

There is a json file created for each job in history server's webDir/jobs/ directory. Such file is not deleted by cleanupExpiredJobs.

And while the cleaned up job is no longer visible in History server's Completed Jobs List in web UI, it can be still accessed on <HistoryServerURL>/#/job/<jobId>/overview.

While this bug probably won't lead to any serious issues, files in history server's folders should be cleaned up thoroughly.

Issue Links

• links to: GitHub Pull Request #12560