[HBASE-26496] [hbase-thirdparty] Exclude the original protobuf-java jar when shading

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: thirdparty-4.0.0, thirdparty-3.5.2
• Component/s: thirdparty
• Hadoop Flags: Reviewed
• Assignee: Duo Zhang
• Reporter: Duo Zhang
• Created: 28/Nov/21 13:25
• Updated: 05/Feb/22 08:18
• Resolved: 29/Nov/21 01:17

Description

We have a patched version of protobuf and we will compile it, and then in shade we also include the original protobuf-java, where we will get lot of warnings

WARNING] hbase-shaded-protobuf-3.5.1.jar, protobuf-java-3.17.1.jar define 670 overlapping class and resource:
[WARNING]   - META-INF/MANIFEST.MF
[WARNING]   - com.google.protobuf.AbstractMessage
[WARNING]   - com.google.protobuf.AbstractMessage$Builder
[WARNING]   - com.google.protobuf.AbstractMessage$BuilderParent
[WARNING]   - com.google.protobuf.AbstractMessageLite
[WARNING]   - com.google.protobuf.AbstractMessageLite$Builder
[WARNING]   - com.google.protobuf.AbstractMessageLite$Builder$LimitedInputStream
[WARNING]   - com.google.protobuf.AbstractMessageLite$InternalOneOfEnum
[WARNING]   - com.google.protobuf.AbstractParser
[WARNING]   - com.google.protobuf.AbstractProtobufList
[WARNING]   - 660 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[WARNING] hbase-shaded-protobuf-3.5.1-sources.jar, protobuf-java-3.17.1-sources.jar define 202 overlapping resources: 
[WARNING]   - META-INF/MANIFEST.MF
[WARNING]   - com/google/protobuf/AbstractMessage.java
[WARNING]   - com/google/protobuf/AbstractMessageLite.java
[WARNING]   - com/google/protobuf/AbstractParser.java
[WARNING]   - com/google/protobuf/AbstractProtobufList.java
[WARNING]   - com/google/protobuf/AllocatedBuffer.java
[WARNING]   - com/google/protobuf/Android.java
[WARNING]   - com/google/protobuf/Any.java
[WARNING]   - com/google/protobuf/AnyOrBuilder.java
[WARNING]   - com/google/protobuf/AnyProto.java
[WARNING]   - 192 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/

Maybe this is the case why we bundle the incorrect protobuf classes and cause the failure when running with JDK8.

Issue Links

• breaks: HBASE-26592 Fix the broken shaded protobuf module
• is related to: HBASE-26492 [branch-2, hbase-thirdparty, Java 8] TestUnloadAccessController and other unit tests fail to start due to ByteBuffer link error
• links to: PR #56