[HADOOP-18487] Make protobuf 2.5 an optional runtime dependency.

• Type: Improvement
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.3.4
• Fix Version/s: 3.3.9
• Component/s: build, ipc
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.3.7
• Hadoop Flags: Incompatible change

Release Note

hadoop modules no longer export protobuf-2.5.0 as a dependency, and it is omitted from the hadoop distribution directory. Applications which use the library must declare an explicit dependency.

Hadoop uses a shaded version of protobuf3 internally, and does not use the 2.5.0 JAR except when compiling compatible classes

• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 10/Oct/22 15:26
• Updated: 19/Mar/24 13:01

Description

uses of protobuf 2.5 and RpcEnginej have been deprecated since 3.3.0 in HADOOP-17046

while still keeping those files around (for a long time...), how about we make the protobuf 2.5.0 export off hadoop common and hadoop-hdfs provided, rather than compile

that way, if apps want it for their own apis, they have to explicitly ask for it, but at least our own scans don't break.

i have no idea what will happen to the rest of the stack at this point, it will be "interesting" to see

Issue Links

• is depended upon by: HADOOP-17860 Upgrade third party protobuf-java-2.5.0.jar to address vulnerabilities #CVE-2015-5237, CVE-2019-15544,
• is related to: HADOOP-13363 Upgrade protobuf from 2.5.0 to something newer
• Testing discovered: YARN-11657 Remove protobuf-2.5 as dependency of hadoop-yarn-api
• links to: GitHub Pull Request #4996
• links to: GitHub Pull Request #6184
• links to: GitHub Pull Request #6185
• links to: GitHub Pull Request #6258