Details
-
Sub-task
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
None
-
jdk-11.0.1
Description
During the review cycle of NIFI-5820, I found that while HTTP S2S works without issue, RAW S2S is failing with following Exception:
2018-12-19 16:19:26,811 ERROR [Site-to-Site Listener] org.apache.nifi.NiFi java.nio.channels.IllegalBlockingModeException: null at java.base/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:121) at org.apache.nifi.remote.SocketRemoteSiteListener$1.run(SocketRemoteSiteListener.java:125) at java.base/java.lang.Thread.run(Thread.java:834)
Despite of the fact that the RAW has been worked with older Java versions, it seems current nio usage at RAW S2S is not correct. And JDK 11 starts complaining about it.
Here are few things I've discovered with current NiFi and nio SocketChannel:
- NiFi accepts RAW S2S client connection with SocketRemoteSiteListener, which uses ServerSocketChannel as non-blocking manner [1]
- But SocketRemoteSiteListener doesn't use Selector API to accept incoming connection and transfer data with the channel. This is the cause of above exception.
- SocketRemoteSiteListener spawns new thread when it accepts connection. This is how connections are handled with a non-nio, standard Socket programming. If we want to use non-blocking NIO, we need to use channels with Selector
- But using non-blocking IO with current NiFi S2S protocol can only add few or none benefit by doing so. [2]
To make RAW S2S work with Java 11, we need either:
A. Stop using nio packages.
B. Implement correct nio usage, meaning use Selector IO and probably we need another thread pool.
I'm going to take the approach A above, because B would take much more refactoring.
[1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/SocketRemoteSiteListener.java#L120
[2] https://stackoverflow.com/questions/12338204/in-java-nio-is-a-selector-useful-for-a-client-socketchannel
