Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-20331 clean up shaded packaging for 2.1
  3. HBASE-20333

break up shaded client into one with no Hadoop and one that's standalone

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • 2.0.0
    • 3.0.0-alpha-1, 2.1.0
    • shading
    • None
    • Hide
      <!-- markdown -->

      Downstream users who need to use both HBase and Hadoop APIs should switch to relying on the new `hbase-shaded-client-byo-hadoop` artifact rather than the existing `hbase-shaded-client` artifact. The new artifact no longer includes and Hadoop classes.

      It should work in combination with either the output of `hadoop classpath` or the Hadoop provided client-facing shaded artifacts in Hadoop 3+.
      Show
      <!-- markdown --> Downstream users who need to use both HBase and Hadoop APIs should switch to relying on the new `hbase-shaded-client-byo-hadoop` artifact rather than the existing `hbase-shaded-client` artifact. The new artifact no longer includes and Hadoop classes. It should work in combination with either the output of `hadoop classpath` or the Hadoop provided client-facing shaded artifacts in Hadoop 3+.

    Description

      there are contexts where we want to stay out of our downstream users way wrt dependencies, but they need more Hadoop classes than we provide. i.e. any downstream client that wants to use both HBase and HDFS in their application, or any non-MR YARN application.

      Now that Hadoop also has shaded client artifacts for Hadoop 3, we're also providing less incremental benefit by including our own rewritten Hadoop classes to avoid downstream needing to pull in all of Hadoop's transitive dependencies.

      right now those users need to ensure that any jars from the Hadoop project are loaded in the classpath prior to our shaded client jar. This is brittle and prone to weird debugging trouble.

      instead, we should have two artifacts: one that just lists Hadoop as a prerequisite and one that still includes the rewritten-but-not-relocated Hadoop classes.

      We can then use docs to emphasize when each of these is appropriate to use.

      Attachments

        1. HBASE-20333.1.patch
          79 kB
          Sean Busbey
        2. HBASE-20333.WIP.0.patch
          84 kB
          Sean Busbey

        Activity

          People

            busbey Sean Busbey
            busbey Sean Busbey
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: