[HUDI-3427] Investigate timeouts in Hudi Kafka Connect Sink

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 1.1.0
• Component/s: kafka-connect
• Epic Link: Hudi Kafka Connect
• Assignee: Unassigned
• Reporter: Ethan Guo
• Created: 14/Feb/22 23:05
• Updated: 21/Dec/23 15:27

Description

When following Quick Start Guide for Hudi Kafka Connect Sink, the sink periodically throws timeout exceptions, though they does not affect delta commits:

[2022-02-14 15:02:39,832] DEBUG [hudi-sink|task-0] ignored: WriteFlusher@349724cf{IDLE}->null (org.apache.hudi.org.eclipse.jetty.io.WriteFlusher:471)
java.nio.channels.ClosedChannelException
	at org.apache.hudi.org.eclipse.jetty.io.WriteFlusher.onClose(WriteFlusher.java:502)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractEndPoint.onClose(AbstractEndPoint.java:353)
	at org.apache.hudi.org.eclipse.jetty.io.ChannelEndPoint.onClose(ChannelEndPoint.java:215)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractEndPoint.doOnClose(AbstractEndPoint.java:225)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:192)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractEndPoint.close(AbstractEndPoint.java:175)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractConnection.onFillInterestedFailed(AbstractConnection.java:174)
	at org.apache.hudi.org.eclipse.jetty.server.HttpConnection.onFillInterestedFailed(HttpConnection.java:503)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractConnection$ReadCallback.failed(AbstractConnection.java:311)
	at org.apache.hudi.org.eclipse.jetty.io.FillInterest.onFail(FillInterest.java:138)
	at org.apache.hudi.org.eclipse.jetty.io.AbstractEndPoint.onIdleExpired(AbstractEndPoint.java:406)
	at org.apache.hudi.org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.apache.hudi.org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[2022-02-14 15:02:39,833] DEBUG [hudi-sink|task-0] onClose FillInterest@1c255bc0{null} (org.apache.hudi.org.eclipse.jetty.io.FillInterest:147)
[2022-02-14 15:02:39,833] DEBUG [hudi-sink|task-0] ignored: WriteFlusher@349724cf{IDLE}->null (org.apache.hudi.org.eclipse.jetty.io.WriteFlusher:471)
java.util.concurrent.TimeoutException: Idle timeout expired: 30003/30000 ms
	at org.apache.hudi.org.eclipse.jetty.io.IdleTimeout.checkIdleTimeout(IdleTimeout.java:166)
	at org.apache.hudi.org.eclipse.jetty.io.IdleTimeout$1.run(IdleTimeout.java:50)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)