[SAMZA-2456] Upgrade RocksDB to 6.5.3

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.3
• Fix Version/s: 1.5
• Assignee: Eric Pan
• Reporter: Eric Pan
• Created: 11/Feb/20 17:58
• Updated: 27/May/20 03:47
• Resolved: 19/Feb/20 02:04

Description

We are currently on RocksDB 5.7.3 version which is over 2 and 1/2 years old build. As such we should upgrade to the latest RocksDB version to include any fixes or new features. Also there is a segment fault when trying to run RocksDB 5.7.3 with Tensorflow described below:

RocksDB version in Samza is 5.7.3<https://jarvis.corp.linkedin.com/codesearch/result/?name=dependency-versions.gradle&path=samza-li%2Fsamza%2Fgradle&reponame=samza%2Fsamza-li>. However, this would cause a segment fault if the dependency order is not properly provisioned. For example, if RockDB library is loaded before Tensorflow library, segment fault is triggered. In contrast, if Tensorflow library is loaded first, then there is no run-time error. Please see the following experiments for more details.

===========experiments==============
public class RocksDBTest{
public static void main(String[] args) throws InterruptedException

{ //System.load("/home/yangzhan/libs/librocksdbjni7259590262328036607.so"); // RocksDB v=5.0.1 => Failed // System.load("/home/yangzhan/libs/librocksdbjni575105474288895339.so"); // RocksDB v=5.7.3 => Failed System.load("/home/yangzhan/libs/librocksdbjni6484631531885492614.so"); // RockdsDB v=5.15.10 => Passed System.load("/home/yangzhan/libs/libtensorflow_framework.so"); }

}
=========================

This issue was reported in a Tensorflow open-source ticket<https://github.com/tensorflow/tensorflow/issues/22307> created by @Shanthoosh Venkataraman<[e-mail]>.

Issue Links

• links to: GitHub Pull Request #1275