[HTTPCORE-758] Requests stuck in IOReactor queue due to fatal exceptions while creating IO dispatchers

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Invalid
• Affects Version/s: 4.4.13
• Component/s: HttpCore NIO
• Environment: Library & version: Apache httpasyncclient - 4.1.4, httpcore: 4.4.13,httpcore-nio: 4.4.10 ; Operating System: Windows ; Runtime Environment: Java 1.8

Language

Java

• Assignee: Unassigned
• Reporter: Karthik Bandi
• Created: 25/Sep/23 05:08
• Updated: 26/Sep/23 14:06
• Resolved: 25/Sep/23 06:39

Description

Hi team,

Requests are stuck in the requestQueue of IO Reactor (DefaultConnectingIOReactor) but not processed due to abnormal termination of I/O Reactor thread that processes the requests (started as part of calling the start() in CloseableHttpAsyncClientBase).

I've enabled http logging and collected the following error logs:

DEBUG 2023-09-18 10:16:38,706Z [parallel_; tid=11876601] [ o.a.http.impl.nio.client.MainClientExec] : [exchange: 47975] start execution
DEBUG 2023-09-18 10:16:38,706Z [parallel_; tid=11876601] [ o.a.h.client.protocol.RequestAddCookies] : CookieSpec selected: standard
DEBUG 2023-09-18 10:16:38,706Z [parallel_; tid=11876601] [ o.a.h.client.protocol.RequestAuthCache] : Auth cache not set in the context
DEBUG 2023-09-18 10:16:38,706Z [parallel_; tid=11876601] [o.a.h.i.n.client.InternalHttpAsyncClient] : [exchange: 47975] Request connection for {s}-> xxxxxxxxxxxxxxxxxxxxx
DEBUG 2023-09-18 10:16:38,706Z [parallel_; tid=11876601] [.n.c.PoolingNHttpClientConnectionManager] : Connection request: [route: {s}->xxxxxxxxxxxxxxxxxxxxxxxxx][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]
ERROR 2023-09-18 10:16:38,757Z [ pool-48041-thread-1] [o.a.h.i.n.client.InternalHttpAsyncClient] : I/O reactor terminated abnormally
org.apache.http.nio.reactor.IOReactorException: Failure opening selector
at org.apache.http.impl.nio.reactor.AbstractIOReactor.<init>(AbstractIOReactor.java:103)
at org.apache.http.impl.nio.reactor.BaseIOReactor.<init>(BaseIOReactor.java:85)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:318)
at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Unable to establish loopback connection
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:94)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:61)
at java.security.AccessController.doPrivileged(Native Method)
at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:171)
at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:50)
at java.nio.channels.Pipe.open(Pipe.java:155)
at sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:142)
at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:44)
at java.nio.channels.Selector.open(Selector.java:227)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.<init>(AbstractIOReactor.java:101)
... 5 common frames omitted
Caused by: java.net.SocketException: Socket operation on nonsocket: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:461)
at sun.nio.ch.Net.bind(Net.java:453)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:85)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:78)
at sun.nio.ch.PipeImpl$Initializer$LoopbackConnector.run(PipeImpl.java:121)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:76)
... 14 common frames omitted

I would like to know what is the cause for underlying exception:

Caused by: java.net.SocketException: Socket operation on nonsocket: bind

Is it environment issue that can be fixed with any configuration changes on Windows system?

If it is non-recoverable exception, why would the HttpAsyncClient doesn't shutdown the connection manager and all relevant components and respond back to the caller by calling the callback methods?

Finally, Can I expect a fix for apache team on this issue? thanks.

Same issue is reported in Apache :: Requests stuck in queue due to termination of I/O Reactor (apachelounge.com)

If there is something wrong in affect versions, please update them.