[FLINK-20713] out/err file support rolling

• Type: New Feature
• Status: Reopened
• Priority: Not a Priority
• Resolution: Unresolved
• Component/s: API / Core, Runtime / Configuration
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: muzimusi
• Created: 22/Dec/20 07:44
• Updated: 08/Oct/21 11:12

Description

It is meaningful for out / err files to support scrolling. In particular, when you have to use stdout , writing out files continuously will eventually fill the disk. Similarly, err files will finally fill the disk when the job is keep rebooting with errors.

Recently, I saw some people asking for 'taskmanager.out rolling' in the community email. We happened to have done this, hoping to help those in need.

The following is our implementation:
    a) Redirect stdout/stderr to logger .
    b) Flinkonyarn/Flinkonk8s/standalone support stdout/stderr redirection。
    c) Provide the default log4J configuration.

 

flinkonyarn.png

Issue Links

• links to: GitHub Pull Request #14476