[FLINK-28283] Improving the log of flink when job start and deploy

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Won't Do
• Affects Version/s: 1.14.2
• Component/s: Runtime / Task
• Assignee: Unassigned
• Reporter: zlzhang0122
• Created: 28/Jun/22 10:17
• Updated: 11/Feb/23 01:41
• Resolved: 11/Feb/23 01:41

Description

When running a large job with many operators and subtasks on flink, the JobManager and TaskManager will have a huge logs about the subtask executing msg such as "XXX switched from CREATED to SCHEDULED、XXX switched from SCHEDULED to DEPLOYING 、XXX switched from DEPLOYING to RUNNING 、XXX switched from RUNNING to CANCELING、XXX switched from CANCELING to CANCELED", etc. .

Maybe we can do some improvement about this, such as aggregate these msg to reduce the log, or change the log level and only logs the failure msg and subtask, etc. Not so sure about the solution, but these msg is really too much.