[FLINK-26314] StreamingCompactingFileSinkITCase.testFileSink failed on azure

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.15.0
• Fix Version/s: 1.15.0
• Component/s: Connectors / FileSystem
• Labels: pull-request-available test-stability
• Assignee: Gen Luo
• Reporter: Yun Gao
• Created: 23/Feb/22 03:23
• Updated: 08/Mar/22 06:11
• Resolved: 08/Mar/22 06:11

Description

Feb 22 13:34:32 [ERROR] Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 12.735 s <<< FAILURE! - in org.apache.flink.connector.file.sink.StreamingCompactingFileSinkITCase
Feb 22 13:34:32 [ERROR] StreamingCompactingFileSinkITCase.testFileSink  Time elapsed: 3.311 s  <<< FAILURE!
Feb 22 13:34:32 java.lang.AssertionError: The record 6788 should occur 4 times,  but only occurs 3time expected:<4> but was:<3>
Feb 22 13:34:32 	at org.junit.Assert.fail(Assert.java:89)
Feb 22 13:34:32 	at org.junit.Assert.failNotEquals(Assert.java:835)
Feb 22 13:34:32 	at org.junit.Assert.assertEquals(Assert.java:647)
Feb 22 13:34:32 	at org.apache.flink.connector.file.sink.utils.IntegerFileSinkTestDataUtils.checkIntegerSequenceSinkOutput(IntegerFileSinkTestDataUtils.java:155)
Feb 22 13:34:32 	at org.apache.flink.connector.file.sink.FileSinkITBase.testFileSink(FileSinkITBase.java:84)
Feb 22 13:34:32 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
Feb 22 13:34:32 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
Feb 22 13:34:32 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
Feb 22 13:34:32 	at java.lang.reflect.Method.invoke(Method.java:498)
Feb 22 13:34:32 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
Feb 22 13:34:32 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
Feb 22 13:34:32 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
Feb 22 13:34:32 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
Feb 22 13:34:32 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
Feb 22 13:34:32 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
Feb 22 13:34:32 	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
Feb 22 13:34:32 	at org.apache.flink.util.TestNameProvider$1.evaluate(TestNameProvider.java:45)
Feb 22 13:34:32 	at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:61)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
Feb 22 13:34:32 	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
Feb 22 13:34:32 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
Feb 22 13:34:32 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
Feb 22 13:34:32 	at org.junit.runners.Suite.runChild(Suite.java:128)
Feb 22 13:34:32 	at org.junit.runners.Suite.runChild(Suite.java:27)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
Feb 22 13:34:32 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
 

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=32023&view=logs&j=d44f43ce-542c-597d-bf94-b0718c71e5e8&t=ed165f3f-d0f6-524b-5279-86f8ee7d0e2d&l=11103

Issue Links

• is caused by: FLINK-26403 SinkWriter should emit all the pending committables on endOfInput
• is caused by: FLINK-26440 CompactorOperatorStateHandler can not work with unaligned checkpoint
• links to: GitHub Pull Request #18916