[FLINK-34911] ChangelogRecoveryRescaleITCase failed fatally with 127 exit code

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 1.20.0
• Component/s: Runtime / State Backends
• Labels: test-stability
• Assignee: Unassigned
• Reporter: Ryan Skraba
• Created: 21/Mar/24 15:02
• Updated: 28/Mar/24 13:44

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=58455&view=logs&j=a657ddbf-d986-5381-9649-342d9c92e7fb&t=dc085d4a-05c8-580e-06ab-21f5624dab16&l=9029

 

Mar 21 01:50:42 01:50:42.553 [ERROR] Command was /bin/sh -c cd '/__w/1/s/flink-tests' && '/usr/lib/jvm/jdk-21.0.1+12/bin/java' '-XX:+UseG1GC' '-Xms256m' '-XX:+IgnoreUnrecognizedVMOptions' '--add-opens=java.base/java.util=ALL-UNNAMED' '--add-opens=java.base/java.io=ALL-UNNAMED' '-Xmx1536m' '-jar' '/__w/1/s/flink-tests/target/surefire/surefirebooter-20240321010847189_810.jar' '/__w/1/s/flink-tests/target/surefire' '2024-03-21T01-08-44_720-jvmRun3' 'surefire-20240321010847189_808tmp' 'surefire_207-20240321010847189_809tmp'
Mar 21 01:50:42 01:50:42.553 [ERROR] Error occurred in starting fork, check output in log
Mar 21 01:50:42 01:50:42.553 [ERROR] Process Exit Code: 127
Mar 21 01:50:42 01:50:42.553 [ERROR] Crashed tests:
Mar 21 01:50:42 01:50:42.553 [ERROR] org.apache.flink.test.checkpointing.ChangelogRecoveryRescaleITCase
Mar 21 01:50:42 01:50:42.553 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:456)
Mar 21 01:50:42 01:50:42.553 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:418)
Mar 21 01:50:42 01:50:42.553 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:297)
Mar 21 01:50:42 01:50:42.553 [ERROR] 	at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:250)
Mar 21 01:50:42 01:50:42.554 [ERROR] 	at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1240)

From the watchdog, only ChangelogRecoveryRescaleITCase didn't complete, specifically parameterized with an EmbeddedRocksDBStateBackend with incremental checkpointing enabled.

The base class (ChangelogRecoveryITCaseBase) starts a MiniClusterWithClientResource

~/Downloads/CI/logs-cron_jdk21-test_cron_jdk21_tests-1710982836$ cat watchdog| grep "Tests run\|Running org.apache.flink" | grep -o "org.apache.flink[^ ]*$" | sort | uniq -c | sort -n | head
      1 org.apache.flink.test.checkpointing.ChangelogRecoveryRescaleITCase
      2 org.apache.flink.api.connector.source.lib.NumberSequenceSourceITCase