[FLINK-19093] "Elasticsearch (v6.3.1) sink end-to-end test" failed with "SubtaskCheckpointCoordinatorImpl was closed without closing asyncCheckpointRunnable 1"

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.12.0
• Fix Version/s: 1.12.0
• Component/s: Runtime / Checkpointing
• Labels: pull-request-available test-stability
• Assignee: Roman Khachatryan
• Reporter: Dian Fu
• Created: 30/Aug/20 02:00
• Updated: 02/Sep/20 10:50
• Resolved: 02/Sep/20 10:50

Description

https://dev.azure.com/apache-flink/apache-flink/_build/results?buildId=5986&view=logs&j=91bf6583-3fb2-592f-e4d4-d79d79c3230a&t=3425d8ba-5f03-540a-c64b-51b8481bf7d6

2020-08-29T22:20:02.3500263Z 2020-08-29 22:20:00,851 INFO  org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable [] - Source: Sequence Source -> Flat Map -> Sink: Unnamed (1/1) - asynchronous part of checkpoint 1 could not be completed.
2020-08-29T22:20:02.3501112Z java.lang.IllegalStateException: SubtaskCheckpointCoordinatorImpl was closed without closing asyncCheckpointRunnable 1
2020-08-29T22:20:02.3502049Z 	at org.apache.flink.util.Preconditions.checkState(Preconditions.java:217) ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
2020-08-29T22:20:02.3503280Z 	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.registerAsyncCheckpointRunnable(SubtaskCheckpointCoordinatorImpl.java:371) ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
2020-08-29T22:20:02.3504647Z 	at org.apache.flink.streaming.runtime.tasks.SubtaskCheckpointCoordinatorImpl.lambda$registerConsumer$2(SubtaskCheckpointCoordinatorImpl.java:479) ~[flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
2020-08-29T22:20:02.3505882Z 	at org.apache.flink.streaming.runtime.tasks.AsyncCheckpointRunnable.run(AsyncCheckpointRunnable.java:95) [flink-dist_2.11-1.12-SNAPSHOT.jar:1.12-SNAPSHOT]
2020-08-29T22:20:02.3506614Z 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_265]
2020-08-29T22:20:02.3507203Z 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_265]
2020-08-29T22:20:02.3507685Z 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_265]
2020-08-29T22:20:02.3509577Z 2020-08-29 22:20:00,927 INFO  org.apache.flink.runtime.taskexecutor.slot.TaskSlotTableImpl [] - Free slot TaskSlot(index:0, state:ACTIVE, resource profile: ResourceProfile{cpuCores=1.0000000000000000, taskHeapMemory=384.000mb (402653174 bytes), taskOffHeapMemory=0 bytes, managedMemory=512.000mb (536870920 bytes), networkMemory=128.000mb (134217730 bytes)}, allocationId: ca890bc4df19c66146370647d07bf510, jobId: 3522a3e4940d4b3cefc6dc1f22123f4b).
2020-08-29T22:20:02.3511425Z 2020-08-29 22:20:00,939 INFO  org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService [] - Remove job 3522a3e4940d4b3cefc6dc1f22123f4b from job leader monitoring.
2020-08-29T22:20:02.3512499Z 2020-08-29 22:20:00,939 INFO  org.apache.flink.runtime.taskexecutor.TaskExecutor           [] - Close JobManager connection for job 3522a3e4940d4b3cefc6dc1f22123f4b.
2020-08-29T22:20:02.3513174Z Checking for non-empty .out files...
2020-08-29T22:20:02.3513706Z No non-empty .out files.
2020-08-29T22:20:02.3513878Z 
2020-08-29T22:20:02.3514679Z [FAIL] 'Elasticsearch (v6.3.1) sink end-to-end test' failed after 0 minutes and 37 seconds! Test exited with exit code 0 but the logs contained errors, exceptions or non-empty .out files
2020-08-29T22:20:02.3515138Z 

Issue Links

• relates to: FLINK-19012 E2E test fails with "Cannot register Closeable, this subtaskCheckpointCoordinator is already closed. Closing argument."
• links to: GitHub Pull Request #13286