Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11656 Classpath isolation for downstream clients
  3. HADOOP-11804

Shaded Hadoop client artifacts and minicluster

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: build
    • Labels:
      None
    • Target Version/s:
    • Release Note:
      Hide
      <!-- markdown -->

      The `hadoop-client` Maven artifact available in 2.x releases pulls
      Hadoop's transitive dependencies onto a Hadoop application's classpath.
      This can be problematic if the versions of these transitive dependencies
      conflict with the versions used by the application.

      [HADOOP-11804](https://issues.apache.org/jira/browse/HADOOP-11804) adds
      new `hadoop-client-api` and `hadoop-client-runtime` artifacts that
      shade Hadoop's dependencies into a single jar. This avoids leaking
      Hadoop's dependencies onto the application's classpath.
      Show
      <!-- markdown --> The `hadoop-client` Maven artifact available in 2.x releases pulls Hadoop's transitive dependencies onto a Hadoop application's classpath. This can be problematic if the versions of these transitive dependencies conflict with the versions used by the application. [ HADOOP-11804 ]( https://issues.apache.org/jira/browse/HADOOP-11804 ) adds new `hadoop-client-api` and `hadoop-client-runtime` artifacts that shade Hadoop's dependencies into a single jar. This avoids leaking Hadoop's dependencies onto the application's classpath.

      Description

      make a hadoop-client-api and hadoop-client-runtime that i.e. HBase can use to talk with a Hadoop cluster without seeing any of the implementation dependencies.

      see proposal on parent for details.

        Attachments

        1. HADOOP-11804.1.patch
          76 kB
          Sean Busbey
        2. HADOOP-11804.10.patch
          102 kB
          Sean Busbey
        3. HADOOP-11804.11.patch
          121 kB
          Sean Busbey
        4. HADOOP-11804.12.patch
          120 kB
          Sean Busbey
        5. HADOOP-11804.13.patch
          121 kB
          Sean Busbey
        6. HADOOP-11804.14.patch
          122 kB
          Sean Busbey
        7. HADOOP-11804.2.patch
          76 kB
          Sean Busbey
        8. HADOOP-11804.3.patch
          82 kB
          Sean Busbey
        9. HADOOP-11804.4.patch
          86 kB
          Sean Busbey
        10. HADOOP-11804.5.patch
          91 kB
          Sean Busbey
        11. HADOOP-11804.6.patch
          96 kB
          Sean Busbey
        12. HADOOP-11804.7.patch
          139 kB
          Sean Busbey
        13. HADOOP-11804.8.patch
          103 kB
          Sean Busbey
        14. HADOOP-11804.9.patch
          103 kB
          Sean Busbey
        15. hadoop-11804-client-test.tar.gz
          1 kB
          Sean Busbey

        Issue Links

        breaks

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-13977 IntelliJ Compilation error in ITUseMiniCluster.java

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-13922 Some modules have dependencies on hadoop-client jar removed by HADOOP-11804

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved
        is blocked by

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-13789 Hadoop Common includes generated test protos in both jar and test-jar

        • Major - Major loss of function.
        • Resolved
        is duplicated by

        Sub-task - The sub-task of the issue HADOOP-13136 shade protobuf in the hadoop-common jar

        • Major - Major loss of function.
        • Resolved
        is related to

        Sub-task - The sub-task of the issue HADOOP-13917 Ensure yetus personality runs the integration tests for the shaded client

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Sub-task - The sub-task of the issue HADOOP-13921 Remove Log4j classes from JobConf

        • Critical - Crashes, loss of data, severe memory leak.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-14284 Shade Guava everywhere

        • Blocker - Blocks development and/or testing work, production could not run
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. YARN-6466 Provide shaded framework jar for containers

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13916 Document how downstream clients should make use of the new shaded client artifacts

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13918 Add integration tests for shaded client based on use by HBase

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13919 add integration tests for shaded client based on use by Spark

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. HADOOP-16822 Provide source artifacts for hadoop-client-api

        • Major - Major loss of function.
        • Patch Available
        relates to

        Bug - A problem which impairs or prevents the functions of the product. HADOOP-12048 many undeclared used dependencies (and declared unused dependencies)

        • Major - Major loss of function.
        • Open
        supercedes

        Sub-task - The sub-task of the issue HADOOP-13136 shade protobuf in the hadoop-common jar

        • Major - Major loss of function.
        • Resolved

          Activity
          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

            • Assignee:
              busbey Sean Busbey
              Reporter:
              busbey Sean Busbey

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment