[FLINK-28977] NullPointerException in HybridSourceSplitEnumerator.close

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.4, 1.15.1
• Fix Version/s: 1.16.0
• Component/s: Connectors / Common
• Labels: pull-request-available pull-requests-available
• Assignee: Michael
• Reporter: Michael
• Created: 15/Aug/22 21:03
• Updated: 30/Aug/22 07:50
• Resolved: 23/Aug/22 12:29

Description

HybridSource pipeline has an intermittent error when reading from s3, usually this error is fixed when pipeline restarts after recovering from checkpoint. But intermittently happens:

2022/08/02 22:26:51.435 INFO  o.a.f.runtime.jobmaster.JobMaster - Trying to recover from a global failure.
org.apache.flink.util.FlinkException: Global failure triggered by OperatorCoordinator for 'Source: hybrid-source -> decrypt -> map2Events -> filterOutNulls -> assignTimestampsAndWatermarks -> logRawJson' (operator fd9fbc680ee884c4eafd0b9c2d3d007f).
at org.apache.flink.runtime.operators.coordination.OperatorCoordinatorHolder$LazyInitializedCoordinatorContext.failJob(OperatorCoordinatorHolder.java:545)
at org.apache.flink.runtime.operators.coordination.RecreateOnResetOperatorCoordinator$DeferrableCoordinator.cleanAndFailJob(RecreateOnResetOperatorCoordinator.java:393)
...
Caused by: java.lang.NullPointerException: null
at org.apache.flink.connector.base.source.hybridspp.HybridSourceSplitEnumerator.close(HybridSourceSplitEnumerator.java:246)
at org.apache.flink.runtime.source.coordinator.SourceCoordinator.close(SourceCoordinator.java:151)
at org.apache.flink.runtime.operators.coordination.ComponentClosingUtils.lambda$closeAsyncWithTimeout$0(ComponentClosingUtils.java:70)
at java.lang.Thread.run(Thread.java:750)
 

Issue Links

• links to: GitHub Pull Request #20587