[FLINK-5408] RocksDB initialization can fail with an UnsatisfiedLinkError in the presence of multiple classloaders

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.0
• Fix Version/s: 1.2.0, 1.3.0
• Component/s: Runtime / State Backends
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 04/Jan/17 23:28
• Updated: 21/Jan/19 22:05
• Resolved: 05/Jan/17 19:39

Description

When the RocksDB is loaded from different ClassLoaders (for example because it is in the user code jar, or loaded dynamically in tests) it may fail with an "java.lang.UnsatisfiedLinkError: Native Library /path/to/temp/dir/librocksdbjni-linux64.so already loaded in another classloader.

Apparently the JVM can handle multiple instances of the same JNI library being loaded in different class loaders, but not when coming from the same file path.

This affects only version 1.2 onward, because from there we extract the JNI library into Flink's temp folders to make sure that it gets cleaned up for example by YARN when the application finishes. When giving a parent directory, RocksDB does not add a unique number sequence to the temp file name.

Issue Links

• is related to: FLINK-11402 User code can fail with an UnsatisfiedLinkError in the presence of multiple classloaders
• links to: GitHub Pull Request #3060