[HADOOP-18198] Release Hadoop 3.3.3: hadoop-3.3.2 with some fixes

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.2
• Fix Version/s: 3.3.3
• Component/s: build
• Labels: pull-request-available
• Target Version/s: 3.3.3
• Assignee: Steve Loughran
• Reporter: Steve Loughran
• Created: 11/Apr/22 13:42
• Updated: 19/Aug/22 09:56
• Resolved: 18/May/22 13:19

Description

Hadoop 3.3.3 is a minor followup release to Hadoop 3.3.2 with all the incremental changes which went in to the 3.2.4 release

• minor CVE fixes in Hadoop source
• CVE fixes in dependencies we know of (protobuf unmarshalling leading to DoS, jackson stack overflow,...)
• replacement of log4j 1.2.17 to reload4j
• node.js update

This is not a release off branch-3.3, it is a fork of 3.3.2 with the changes.

The next release of branch-3.3 will be numbered hadoop-3.3.4; updating maven versions and JIRA fix versions is part of this release process.

The changes here are already in branch 3.2.4; this completes the set

CVEs fixed

• CVE-2022-26612: Apache Hadoop: Arbitrary file write in FileUtil#unpackEntries on Windows (HADOOP-18155)
• CVE-2022-25168 Verify FileUtils.unTar() handling of missing .tar files. (HADOOP-18136)

Issue Links

• contains: HADOOP-18212 hadoop-client-runtime latest version 3.3.2 has security issues
• depends upon: HDFS-16422 Fix thread safety of EC decoding during concurrent preads
• depends upon: HDFS-16437 ReverseXML processor doesn't accept XML files without the SnapshotDiffSection.
• depends upon: HDFS-16507 [SBN read] Avoid purging edit log which is in progress
• depends upon: YARN-10720 YARN WebAppProxyServlet should support connection timeout to prevent proxy server from hanging
• depends upon: HADOOP-17650 Fails to build using Maven 3.8.1
• depends upon: HADOOP-18109 Ensure that default permissions of directories under internal ViewFS directories are the same as directories on target filesystems
• depends upon: HADOOP-18160 `org.wildfly.openssl` should not be shaded by Hadoop build
• depends upon: HADOOP-18178 Upgrade jackson to 2.13.2 and jackson-databind to 2.13.2.2. CVE-2020-36518
• depends upon: HDFS-16428 Source path with storagePolicy cause wrong typeConsumed while rename
• depends upon: YARN-11014 YARN incorrectly validates maximum capacity resources on the validation API
• depends upon: YARN-11075 Explicitly declare serialVersionUID in LogMutation class
• depends upon: HADOOP-17341 Upgrade commons-codec to 1.15
• depends upon: HDFS-11041 Unable to unregister FsDatasetState MBean if DataNode is shutdown twice
• depends upon: HADOOP-18088 Replace log4j 1.x with reload4j
• depends upon: HDFS-16501 Print the exception when reporting a bad block
• depends upon: MAPREDUCE-7373 Building MapReduce NativeTask fails on Fedora 34+
• depends upon: HADOOP-18214 Update BUILDING.txt
• depends upon: HDFS-16355 Improve the description of dfs.block.scanner.volume.bytes.per.second
• depends upon: HADOOP-18155 Refactor tests in TestFileUtil
• depends upon: HADOOP-18125 Utility to identify git commit / Jira fixVersion discrepancies for RC preparation
• is blocked by: HADOOP-18202 create-release fails fatal: unsafe repository ('/build/source' is owned by someone else)
• is related to: HADOOP-18293 Release Hadoop 3.3.4 critical fix update
• is related to: HADOOP-18305 Release Hadoop 3.3.4: minor update of hadoop-3.3.3
• relates to: HADOOP-18290 Fix some compatibility issues with 3.3.3 release notes
• links to: GitHub Pull Request #4164
• links to: GitHub Pull Request #4202
• links to: GitHub Pull Request #4239

Sub-Tasks

• create release docker builds failing with network issues retrieving artifacts: Open