Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5174 NiFi Compatibility with Java 11
  3. NIFI-5952

RAW Site-to-Site fails with java.nio.channels.IllegalBlockingModeException

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 1.10.0
    • Core Framework
    • jdk-11.0.1

    Description

      During the review cycle of NIFI-5820, I found that while HTTP S2S works without issue, RAW S2S is failing with following Exception:

       2018-12-19 16:19:26,811 ERROR [Site-to-Site Listener] org.apache.nifi.NiFi
       java.nio.channels.IllegalBlockingModeException: null
       at java.base/sun.nio.ch.ServerSocketAdaptor.accept(ServerSocketAdaptor.java:121)
       at org.apache.nifi.remote.SocketRemoteSiteListener$1.run(SocketRemoteSiteListener.java:125)
       at java.base/java.lang.Thread.run(Thread.java:834)
      

      Despite of the fact that the RAW has been worked with older Java versions, it seems current nio usage at RAW S2S is not correct. And JDK 11 starts complaining about it.

      Here are few things I've discovered with current NiFi and nio SocketChannel:

      • NiFi accepts RAW S2S client connection with SocketRemoteSiteListener, which uses ServerSocketChannel as non-blocking manner [1]
      • But SocketRemoteSiteListener doesn't use Selector API to accept incoming connection and transfer data with the channel. This is the cause of above exception.
      • SocketRemoteSiteListener spawns new thread when it accepts connection. This is how connections are handled with a non-nio, standard Socket programming. If we want to use non-blocking NIO, we need to use channels with Selector
      • But using non-blocking IO with current NiFi S2S protocol can only add few or none benefit by doing so. [2]

      To make RAW S2S work with Java 11, we need either:
      A. Stop using nio packages.
      B. Implement correct nio usage, meaning use Selector IO and probably we need another thread pool.

      I'm going to take the approach A above, because B would take much more refactoring.

      [1] https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-site-to-site/src/main/java/org/apache/nifi/remote/SocketRemoteSiteListener.java#L120
      [2] https://stackoverflow.com/questions/12338204/in-java-nio-is-a-selector-useful-for-a-client-socketchannel

      Attachments

        Issue Links

          Activity

            People

              ijokarumawak Koji Kawamura
              ijokarumawak Koji Kawamura
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 10m
                  2h 10m