[HBASE-18271] Shade netty - ASF JIRA

Voters

Watch issue

Watchers

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Resolved
Priority: Blocker
Resolution: Fixed
Affects Version/s: 2.0.0
Fix Version/s: 2.0.0
Component/s: rpc
Labels:
None

Release Note:

Hide
Depend on hbase-thirdparty for our netty instead of directly relying on netty-all. netty is relocated in hbase-thirdparty from io.netty to org.apache.hadoop.hbase.shaded.io.netty. One kink is that netty bundles an .so. Its files also are relocated. So netty can find the .so content, need to specify on command-line a system property telling netty about the shading.

The .so trick is from
https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin

In essence we need the below defined whenever we run tests or deploy:

-Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded.

(The trailing '.' is required)

See toward the end of this issue for how to pass config: https://github.com/netty/netty/issues/6665

The system property has been added to bin/hbase. If starting hbase with other than bin/hbase, add this system property (at least on linux).

For devs, going forward, do not reference io.netty. Reference org.apache.hadoop.hbase.io.netty instead. Here is sample:

{code}
-import io.netty.channel.Channel;
-import io.netty.channel.EventLoop;
+import org.apache.hadoop.hbase.shaded.io.netty.channel.Channel;
+import org.apache.hadoop.hbase.shaded.io.netty.channel.EventLoop;
{code}

Show
Depend on hbase-thirdparty for our netty instead of directly relying on netty-all. netty is relocated in hbase-thirdparty from io.netty to org.apache.hadoop.hbase.shaded.io.netty. One kink is that netty bundles an .so. Its files also are relocated. So netty can find the .so content, need to specify on command-line a system property telling netty about the shading. The .so trick is from https://stackoverflow.com/questions/33825743/rename-files-inside-a-jar-using-some-maven-plugin In essence we need the below defined whenever we run tests or deploy: -Dorg.apache.hadoop.hbase.shaded.io.netty.packagePrefix=org.apache.hadoop.hbase.shaded. (The trailing '.' is required) See toward the end of this issue for how to pass config: https://github.com/netty/netty/issues/6665 The system property has been added to bin/hbase. If starting hbase with other than bin/hbase, add this system property (at least on linux). For devs, going forward, do not reference io.netty. Reference org.apache.hadoop.hbase.io.netty instead. Here is sample: {code} -import io.netty.channel.Channel; -import io.netty.channel.EventLoop; +import org.apache.hadoop.hbase.shaded.io.netty.channel.Channel; +import org.apache.hadoop.hbase.shaded.io.netty.channel.EventLoop; {code}

Description

Our new prefatory project, hbase-thirdparty, includes a relocated netty 4.1.12. This issue is about changing references in hbase to make use of this shaded netty. This way we will take ourselves out of the clashing library saga and change netty as we see fit.

One kink is the inclusion inside the netty-all jar of an .so. We need to make sure it gets loaded and that on linux we are doing native epoll; all should be in place to do this but need to add a system property to bin/hbase for netty to pick up. TODO as part of this issue. See tail of upgrade guava sibling issue.