Cassandra
  1. Cassandra
  2. CASSANDRA-1908

Implement the CLibrary using JNI module to avoid the LGPL dependency on JNA

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Not a Problem
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      Cassandra can't ship JNA out of the box since it's LGPL licensed, so many of the performance optimizing features in the CLibrary class are not available in a simple install. It should be trivial to implement a real JNI library for the CLibrary class.

      1. cassandra-jni.zip
        20 kB
        Hiram Chirino

        Issue Links

          Activity

          Hide
          Hiram Chirino added a comment -

          Attaching a port of the latest CLibrary. Build with maven.
          It code generates the JNI library implementation. See readme.md for more details.

          Show
          Hiram Chirino added a comment - Attaching a port of the latest CLibrary. Build with maven. It code generates the JNI library implementation. See readme.md for more details.
          Hide
          Jonathan Ellis added a comment -

          Thanks, Hiram!

          Personally, I prefer the inconvenience of an LGPL library (which is largely mitigated by deb/rpm packaging) to the inconvenience of maintaining a maven + autoconf build for a native extension, but I'm open to exploring the options.

          Show
          Jonathan Ellis added a comment - Thanks, Hiram! Personally, I prefer the inconvenience of an LGPL library (which is largely mitigated by deb/rpm packaging) to the inconvenience of maintaining a maven + autoconf build for a native extension, but I'm open to exploring the options.
          Hide
          Eric Evans added a comment - - edited

          It might be trivial to implement a JNI library, but I don't think it's going to be trivial to build architecture dependent code at release time, field issues, debug, etc, for N platforms.

          I'm also open to alternatives, but I'd need to be convinced that the cure wasn't worse than the disease.

          Show
          Eric Evans added a comment - - edited It might be trivial to implement a JNI library, but I don't think it's going to be trivial to build architecture dependent code at release time, field issues, debug, etc, for N platforms. I'm also open to alternatives, but I'd need to be convinced that the cure wasn't worse than the disease.
          Hide
          Hiram Chirino added a comment -

          The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right.

          Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance

          But yeah, if you want to include a few pre-built JNI libraries, for example linux 32/64 and OS X, it's release process gets more complicated as you now have to build the jni modules on a couple of platforms. To keep that effort to minimum I'd recommend you guys independently release the jni module and then just include it as part of the cassandra distribution. Once it stabilizes, you probably won't be releasing it very often.

          Show
          Hiram Chirino added a comment - The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right. Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance But yeah, if you want to include a few pre-built JNI libraries, for example linux 32/64 and OS X, it's release process gets more complicated as you now have to build the jni modules on a couple of platforms. To keep that effort to minimum I'd recommend you guys independently release the jni module and then just include it as part of the cassandra distribution. Once it stabilizes, you probably won't be releasing it very often.
          Hide
          Peter Schuller added a comment -

          I was thinking along those lines (separate project) before when there was some JNA vs. JNI debate w.r.t. direct I/O. A separate project could that provides some fairly simple and specific things that tend to be useful in a pragmatic way, without trying to be overly formal or a complete posix wrapper. It could be useful for others except Cassandra, and it would remove any build hassle trade-offs in the JNA vs. JNI decision from Cassandra.

          Also, for future work that might imply very frequent mincore() calls, the calling overhead, if the numbers claimed are correct, could possibly be significant for Cassandra (although I'm pretty paranoid about the cost of mincore() to begin with, which might dwarf the JNI vs. JNA issue).

          Show
          Peter Schuller added a comment - I was thinking along those lines (separate project) before when there was some JNA vs. JNI debate w.r.t. direct I/O. A separate project could that provides some fairly simple and specific things that tend to be useful in a pragmatic way, without trying to be overly formal or a complete posix wrapper. It could be useful for others except Cassandra, and it would remove any build hassle trade-offs in the JNA vs. JNI decision from Cassandra. Also, for future work that might imply very frequent mincore() calls, the calling overhead, if the numbers claimed are correct, could possibly be significant for Cassandra (although I'm pretty paranoid about the cost of mincore() to begin with, which might dwarf the JNI vs. JNA issue).
          Hide
          Eric Evans added a comment -

          The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right.

          Sure, but someone else is doing that work for us. And as Jonathan mentioned, JNA is package for all architectures on the most common platforms.

          Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance

          Performance is always a good argument to make. Do you have before/after numbers to show that it will make a substantial difference for us?

          Show
          Eric Evans added a comment - The JNA solution has all the same architecture dependent code for N platform problems, what's worse is that it's using a FFI and your hoping that everything maps out right. Sure, but someone else is doing that work for us. And as Jonathan mentioned, JNA is package for all architectures on the most common platforms. Additionally, because of the FFI, the JNI interface is going to be orders of magnitude more efficent. See: https://jna.dev.java.net/#performance Performance is always a good argument to make. Do you have before/after numbers to show that it will make a substantial difference for us?
          Hide
          Jonathan Ellis added a comment -

          The page Hiram linked says "JNA supports a direct mapping method which can improve performance substantially, approaching that of custom JNI." This is the approach we are using (Native.register, instead of the slower Native.loadLibrary).

          Show
          Jonathan Ellis added a comment - The page Hiram linked says "JNA supports a direct mapping method which can improve performance substantially, approaching that of custom JNI." This is the approach we are using (Native.register, instead of the slower Native.loadLibrary).
          Hide
          Hiram Chirino added a comment -

          JNA does not really do the work of validating that the native method are properly mapped. Only testing on the actual platforms can do that. JNA just helps avoid pain of doing muti-platform builds (which is nice), but if you want to test on the actual platforms then you might as well do a platform specific build too.

          The performance of the interface is only going to be critical if it's in a performance hot spot which right now does not seem to be the case.

          Show
          Hiram Chirino added a comment - JNA does not really do the work of validating that the native method are properly mapped. Only testing on the actual platforms can do that. JNA just helps avoid pain of doing muti-platform builds (which is nice), but if you want to test on the actual platforms then you might as well do a platform specific build too. The performance of the interface is only going to be critical if it's in a performance hot spot which right now does not seem to be the case.
          Hide
          Stephen Connolly added a comment -

          You could always host the JNI project on http://apache-extras.org/ and as long as it uses ASLv2 and if it publishes to Maven Central it should be possible to pull the build artifacts down... that would decouple the build and release of the JNI code from the build and release of apache cassandra

          Show
          Stephen Connolly added a comment - You could always host the JNI project on http://apache-extras.org/ and as long as it uses ASLv2 and if it publishes to Maven Central it should be possible to pull the build artifacts down... that would decouple the build and release of the JNI code from the build and release of apache cassandra
          Hide
          Jonathan Ellis added a comment -

          As above, we're pretty happy with how JNA is working for us.

          Show
          Jonathan Ellis added a comment - As above, we're pretty happy with how JNA is working for us.

            People

            • Assignee:
              Unassigned
              Reporter:
              Hiram Chirino
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development