[FLINK-22050] Failed to release input gate (IllegalReferenceCountException)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13.0
• Fix Version/s: 1.13.0
• Component/s: Runtime / Network
• Labels: pull-request-available
• Assignee: Roman Khachatryan
• Reporter: Roman Khachatryan
• Created: 30/Mar/21 14:16
• Updated: 28/Aug/21 11:14
• Resolved: 06/Apr/21 09:17

Description

When running UnalignedCheckpointITCase continuously, this error is periodically reported (though tests pass):

[INFO] Running org.apache.flink.test.checkpointing.UnalignedCheckpointITCase
34948 [Canceler for keyed (19/20)#1 (532afce89e3c90301e37b2f5e9a0bded).] ERROR org.apache.flink.runtime.taskmanager.Task [] - Failed to release input gate for task keyed (19/20)#1.
org.apache.flink.shaded.netty4.io.netty.util.IllegalReferenceCountException: refCnt: 0, decrement: 1
        at org.apache.flink.shaded.netty4.io.netty.util.internal.ReferenceCountUpdater.toLiveRealRefCnt(ReferenceCountUpdater.java:74) ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
        at org.apache.flink.shaded.netty4.io.netty.util.internal.ReferenceCountUpdater.release(ReferenceCountUpdater.java:138) ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
        at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractReferenceCountedByteBuf.release(AbstractReferenceCountedByteBuf.java:100) ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
        at org.apache.flink.runtime.io.network.buffer.NetworkBuffer.recycleBuffer(NetworkBuffer.java:156) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.BufferManager.releaseAllBuffers(BufferManager.java:234) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.releaseAllResources(RemoteInputChannel.java:267) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.close(SingleInputGate.java:555) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.close(InputGateWithMetrics.java:119) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.Task.closeNetworkResources(Task.java:978) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.Task$TaskCanceler.run(Task.java:1559) [classes/:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]
38445 [Canceler for keyed (19/20)#2 (f94480c534db164b2cdf7f44682593d1).] ERROR org.apache.flink.runtime.taskmanager.Task [] - Failed to release input gate for task keyed (19/20)#2.
org.apache.flink.shaded.netty4.io.netty.util.IllegalReferenceCountException: refCnt: 0
        at org.apache.flink.shaded.netty4.io.netty.buffer.AbstractByteBuf.ensureAccessible(AbstractByteBuf.java:1489) ~[flink-shaded-netty-4.1.49.Final-12.0.jar:?]
        at org.apache.flink.runtime.io.network.buffer.NetworkBuffer.getMemorySegment(NetworkBuffer.java:139) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.BufferManager.releaseAllBuffers(BufferManager.java:232) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.RemoteInputChannel.releaseAllResources(RemoteInputChannel.java:267) ~[classes/:?]
        at org.apache.flink.runtime.io.network.partition.consumer.SingleInputGate.close(SingleInputGate.java:555) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.InputGateWithMetrics.close(InputGateWithMetrics.java:119) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.Task.closeNetworkResources(Task.java:978) ~[classes/:?]
        at org.apache.flink.runtime.taskmanager.Task$TaskCanceler.run(Task.java:1559) [classes/:?]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_282]

Issue Links

• links to: GitHub Pull Request #15440
• links to: GitHub Pull Request #15460