Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.0
    • Fix Version/s: 2.0.0
    • Component/s: rpc
    • Labels:
      None
    • Release Note:
      Hide
      Depend on hbase-thirdparty for our netty instead of directly relying on netty-all. netty is relocated in hbase-thirdparty from io.netty to org.apache.hadoop.hbase.shaded.io.netty. One kink is that netty bundles an .so. Its files also are relocated. So netty can find the .so content, need to specify on command-line a system property telling netty about the shading.

      The .so trick is from
                   https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin

      In essence we need the below defined whenever we run tests or deploy:

      -Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded.

      (The trailing '.' is required)

      See toward the end of this issue for how to pass config: https://github.com/netty/netty/issues/6665

      The system property has been added to bin/hbase. If starting hbase with other than bin/hbase, add this system property (at least on linux).

      For devs, going forward, do not reference io.netty. Reference org.apache.hadoop.hbase.io.netty instead. Here is sample:

      {code}
      -import io.netty.channel.Channel;
      -import io.netty.channel.EventLoop;
      +import org.apache.hadoop.hbase.shaded.io.netty.channel.Channel;
      +import org.apache.hadoop.hbase.shaded.io.netty.channel.EventLoop;
      {code}
      Show
      Depend on hbase-thirdparty for our netty instead of directly relying on netty-all. netty is relocated in hbase-thirdparty from io.netty to org.apache.hadoop.hbase.shaded.io.netty. One kink is that netty bundles an .so. Its files also are relocated. So netty can find the .so content, need to specify on command-line a system property telling netty about the shading. The .so trick is from               https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin In essence we need the below defined whenever we run tests or deploy: -Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded. (The trailing '.' is required) See toward the end of this issue for how to pass config: https://github.com/netty/netty/issues/6665 The system property has been added to bin/hbase. If starting hbase with other than bin/hbase, add this system property (at least on linux). For devs, going forward, do not reference io.netty. Reference org.apache.hadoop.hbase.io.netty instead. Here is sample: {code} -import io.netty.channel.Channel; -import io.netty.channel.EventLoop; +import org.apache.hadoop.hbase.shaded.io.netty.channel.Channel; +import org.apache.hadoop.hbase.shaded.io.netty.channel.EventLoop; {code}

      Description

      Our new prefatory project, hbase-thirdparty, includes a relocated netty 4.1.12. This issue is about changing references in hbase to make use of this shaded netty. This way we will take ourselves out of the clashing library saga and change netty as we see fit.

      One kink is the inclusion inside the netty-all jar of an .so. We need to make sure it gets loaded and that on linux we are doing native epoll; all should be in place to do this but need to add a system property to bin/hbase for netty to pick up. TODO as part of this issue. See tail of upgrade guava sibling issue.

        Attachments

        1. HBASE-18271.master.001.patch
          77 kB
          stack
        2. HBASE-18271.master.002.patch
          77 kB
          stack
        3. HBASE-18271-Shade-netty-Purge-mention-of-netty-all.003.patch
          79 kB
          stack
        4. HBASE-18271.master.003.patch
          80 kB
          stack
        5. HBASE-18271.master.004.patch
          81 kB
          stack
        6. HBASE-18271.master.005.patch
          81 kB
          stack

          Issue Links

            Activity

              People

              • Assignee:
                stack stack
                Reporter:
                stack stack
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: