[FLINK-24169] Flaky local YARN tests relying on log files - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: 1.20.0
Component/s: Deployment / YARN
Labels:

Description

While working on PR #16989 for ~~FLINK-23611~~, we experienced some flakiness when running YARNSessionCapacitySchedulerITCase.testDetachedPerJobYarnCluster locally.

David Morávek discovered a bug in log4j (see LOG4J2-3155). The bug affects the test because they check the log files for specific log messages. The log messages ends up in the wrong log file if the rolling update mechanism is trigger. This does not seem to be an issue on AzureCI due to the slower hardware used for the worker machines.

A solution to overcome this issue would be to add a custom log4j configuration that disables the appender.main.policies.startup.type = OnStartupTriggeringPolicy which is present in flink-dist's log4j configuration.