[FLINK-11782] Enhance TaskManager log visualization by listing all log files for Flink web UI

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: Runtime / Web Frontend
• Assignee: vinoyang
• Reporter: vinoyang
• Created: 28/Feb/19 11:08
• Updated: 29/Dec/20 10:28
• Resolved: 29/Dec/20 10:28

Description

Currently, there are some features to be enhanced when Flink web UI displays TaskManager logs:

1) Flink web UI can only display the log of task manager and stdout;
2) Flink web UI can only display the latest one log file;

In the log folder of a Task Manager, we usually have more files, including GC log files, multiple files generated based on the rolling rules of the logging framework (time or log file size), user's business log files (We internally implemented the separation of Flink from the user log). For these files, the current Flink web UI cannot be displayed.

I personally suggest that we can refer to the way YARN displays the container's log (see attachment 1), first list all the log files (see attachment 2), and then click to display the files of these logs.

 

This feature can be independent or part of the log separation (see FLINK-11202).

It has now been added to the design documentation for Log separation: https://docs.google.com/document/d/1TTYAtFoTWaGCveKDZH394FYdRyNyQFnVoW5AYFvnr5I/edit?usp=sharing

Issue Links

• duplicates: FLINK-13987 Better TM/JM Log Display
• is related to: FLINK-13987 Better TM/JM Log Display