[FLINK-13172] JVM crash with dynamic netty-tcnative wrapper to openSSL on some OS

• Type: Bug
• Status: Open
• Priority: Not a Priority
• Resolution: Unresolved
• Affects Version/s: 1.9.0
• Component/s: Runtime / Network, Tests
• Labels: auto-deprioritized-major auto-unassigned
• Assignee: Unassigned
• Reporter: Nico Kruber
• Created: 09/Jul/19 16:06
• Updated: 31/Aug/21 13:48

Description

The dynamically-linked wrapper library in flink-shaded-netty-tcnative-dynamic may not work on all systems, depending on how the system-provided openSSL library is built.
As a result, when trying to run Flink with security.ssl.provider: OPENSSL or just running a test based on SSLUtilsTest (which checks for openSSL availability which is enough to trigger the error below), the JVM will crash, e.g. with

• on SUSE-based systems:

  /usr/lib64/jvm/java-openjdk/bin/java: relocation error: /tmp/liborg_apache_flink_shaded_netty4_netty_tcnative_linux_x86_644115489043239307863.so: symbol TLSv1_2_server_method version OPENSSL_1.0.1 not defined in file libssl.so.1.0.0 with link time reference
  

• on Arch Linux:

  /usr/lib/jvm/default/bin/java: relocation error: /tmp/liborg_apache_flink_shaded_netty4_netty_tcnative_linux_x86_648476498532937980008.so: symbol SSLv3_method version OPENSSL_1.0.0 not defined in file libssl.so.1.0.0 with link time reference
  

Possible solutions:

1. build your own OS-dependent dynamically-linked netty-tcnative library and shade it in your own build of flink-shaded-netty-tcnative-dynamic, or
2. use flink-shaded-netty-tcnative-static:

   git clone https://github.com/apache/flink-shaded.git
   cd flink-shaded
   mvn clean package -Pinclude-netty-tcnative-static -pl flink-shaded-netty-tcnative-static
   

3. get your OS-dependent build into netty-tcnative as a special branch similar to what they currently do with Fedora-based systems

Issue Links

• is related to: FLINK-23468 Benchmarks networkThroughput for OpenSSL do not run
• links to: Upstream issue

Sub-Tasks

• Only run openSSL tests if desired: Closed