Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-3458

Support Hadoop 2.x with build-time classpath separation instead of isolated classloader

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 1.0.0
    • build

    Description

      Apache Hadoop Ozone is a Hadoop subproject. It depends on the released Hadoop 3.2. But as Hadoop 3.2 is very rare in production, older versions should be supported to make it possible to work together with Spark, Hive, HBase and older clusters.

      Our current approach is using classloader based separation (ozonefs "legacy" jar), which has multiple problems:

      1. It's quite complex and hard to debug
      2. It couldn't work together with security

      The issue proposes to use a different approach
      1. Reduce the dependency on Hadoop (including the replacement of hadoop metrics and cleanup of the usage of configuration)
      2. Create multiple version from ozonefs-client with different compile time dependency.

      Attachments

        1. classpath.pdf
          312 kB
          Marton Elek

        Issue Links

          Activity

            People

              elek Marton Elek
              elek Marton Elek
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: