[HDDS-3458] Support Hadoop 2.x with build-time classpath separation instead of isolated classloader

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: build
• Labels: Triaged
• Target Version/s: 1.0.0
• Assignee: Marton Elek
• Reporter: Marton Elek
• Created: 20/Apr/20 07:47
• Updated: 25/Jun/20 08:42
• Resolved: 25/Jun/20 08:42

Description

Apache Hadoop Ozone is a Hadoop subproject. It depends on the released Hadoop 3.2. But as Hadoop 3.2 is very rare in production, older versions should be supported to make it possible to work together with Spark, Hive, HBase and older clusters.

Our current approach is using classloader based separation (ozonefs "legacy" jar), which has multiple problems:

1. It's quite complex and hard to debug
2. It couldn't work together with security

The issue proposes to use a different approach
1. Reduce the dependency on Hadoop (including the replacement of hadoop metrics and cleanup of the usage of configuration)
2. Create multiple version from ozonefs-client with different compile time dependency.

Issue Links

• supercedes: HDDS-2881 Legacy (isolated) ozonefs couldn't work in secure environments
• supercedes: HDDS-1552 RpcClient works with both Hadoop-3 and Hadoop-2