Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11656 Classpath isolation for downstream clients
  3. HADOOP-11804

Shaded Hadoop client artifacts and minicluster

          Details

          • Type: Sub-task
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 3.0.0-alpha2
          • Component/s: build
          • Labels:
            None
          • Target Version/s:
          • Release Note:
            Hide
            <!-- markdown -->

            The `hadoop-client` Maven artifact available in 2.x releases pulls
            Hadoop's transitive dependencies onto a Hadoop application's classpath.
            This can be problematic if the versions of these transitive dependencies
            conflict with the versions used by the application.

            [HADOOP-11804](https://issues.apache.org/jira/browse/HADOOP-11804) adds
            new `hadoop-client-api` and `hadoop-client-runtime` artifacts that
            shade Hadoop's dependencies into a single jar. This avoids leaking
            Hadoop's dependencies onto the application's classpath.
            Show
            <!-- markdown --> The `hadoop-client` Maven artifact available in 2.x releases pulls Hadoop's transitive dependencies onto a Hadoop application's classpath. This can be problematic if the versions of these transitive dependencies conflict with the versions used by the application. [ HADOOP-11804 ]( https://issues.apache.org/jira/browse/HADOOP-11804 ) adds new `hadoop-client-api` and `hadoop-client-runtime` artifacts that shade Hadoop's dependencies into a single jar. This avoids leaking Hadoop's dependencies onto the application's classpath.

            Description

            make a hadoop-client-api and hadoop-client-runtime that i.e. HBase can use to talk with a Hadoop cluster without seeing any of the implementation dependencies.

            see proposal on parent for details.

              Attachments

              1. HADOOP-11804.14.patch
                122 kB
                Sean Busbey
              2. HADOOP-11804.13.patch
                121 kB
                Sean Busbey
              3. HADOOP-11804.12.patch
                120 kB
                Sean Busbey
              4. HADOOP-11804.11.patch
                121 kB
                Sean Busbey
              5. HADOOP-11804.10.patch
                102 kB
                Sean Busbey
              6. hadoop-11804-client-test.tar.gz
                1 kB
                Sean Busbey
              7. HADOOP-11804.9.patch
                103 kB
                Sean Busbey
              8. HADOOP-11804.8.patch
                103 kB
                Sean Busbey
              9. HADOOP-11804.7.patch
                139 kB
                Sean Busbey
              10. HADOOP-11804.6.patch
                96 kB
                Sean Busbey
              11. HADOOP-11804.5.patch
                91 kB
                Sean Busbey
              12. HADOOP-11804.4.patch
                86 kB
                Sean Busbey
              13. HADOOP-11804.3.patch
                82 kB
                Sean Busbey
              14. HADOOP-11804.2.patch
                76 kB
                Sean Busbey
              15. HADOOP-11804.1.patch
                76 kB
                Sean Busbey

                Issue Links

                breaks

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-13977 IntelliJ Compilation error in ITUseMiniCluster.java

                • Major - Major loss of function.
                • Open

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-13922 Some modules have dependencies on hadoop-client jar removed by HADOOP-11804

                • Blocker - Blocks development and/or testing work, production could not run
                • Resolved
                is blocked by

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-13789 Hadoop Common includes generated test protos in both jar and test-jar

                • Major - Major loss of function.
                • Resolved
                is duplicated by

                Sub-task - The sub-task of the issue HADOOP-13136 shade protobuf in the hadoop-common jar

                • Major - Major loss of function.
                • Resolved
                is related to

                Sub-task - The sub-task of the issue HADOOP-13917 Ensure yetus personality runs the integration tests for the shaded client

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                Sub-task - The sub-task of the issue HADOOP-13921 Remove Log4j classes from JobConf

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-14284 Shade Guava everywhere

                • Blocker - Blocks development and/or testing work, production could not run
                • Resolved

                New Feature - A new feature of the product, which has yet to be developed. YARN-6466 Provide shaded framework jar for containers

                • Major - Major loss of function.
                • Open

                Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13916 Document how downstream clients should make use of the new shaded client artifacts

                • Major - Major loss of function.
                • Open

                Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13918 Add integration tests for shaded client based on use by HBase

                • Minor - Minor loss of function, or other problem where easy workaround is present.
                • Open

                Improvement - An improvement or enhancement to an existing feature or task. HADOOP-13919 add integration tests for shaded client based on use by Spark

                • Minor - Minor loss of function, or other problem where easy workaround is present.
                • Open
                relates to

                Bug - A problem which impairs or prevents the functions of the product. HADOOP-12048 many undeclared used dependencies (and declared unused dependencies)

                • Major - Major loss of function.
                • Open
                supercedes

                Sub-task - The sub-task of the issue HADOOP-13136 shade protobuf in the hadoop-common jar

                • Major - Major loss of function.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      busbey Sean Busbey
                      Reporter:
                      busbey Sean Busbey
                    • Votes:
                      1 Vote for this issue
                      Watchers:
                      39 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: