[HADOOP-18160] `org.wildfly.openssl` should not be shaded by Hadoop build

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.1
• Fix Version/s: 3.4.0, 3.3.3
• Component/s: build
• Labels: pull-request-available

Environment

hadoop 3.3.1

spark 3.2.1

JDK8

• Assignee: André F.
• Reporter: André F.
• Created: 16/Mar/22 10:50
• Updated: 09/Jul/22 15:41
• Resolved: 18/Apr/22 15:31

Description

`org.wildfly.openssl` is a runtime library and its references are being shaded on Hadoop, breaking the integration with other frameworks like Spark, whenever the "fs.s3a.ssl.channel.mode" is set to "openssl". The error produced in this situation is:

Suppressed: java.lang.NoClassDefFoundError: org/apache/hadoop/shaded/org/wildfly/openssl/OpenSSLProvider

Whenever it tries to be instantiated from the `DelegatingSSLSocketFactory`. Spark tries to add it to its classpath without the shade, thus creating this issue.

Dependencies which are not on "compile" scope should probably not be shaded to avoid this kind of integration issues.

 

Issue Links

• duplicates: HADOOP-18331 Shouldn't relocate org/wildfly/openssl in shaded client
• is depended upon by: HADOOP-18198 Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes
• is related to: SPARK-38330 Certificate doesn't match any of the subject alternative names: [*.s3.amazonaws.com, s3.amazonaws.com]
• links to: GitHub Pull Request #4074