[HADOOP-13382] remove unneeded commons-httpclient dependencies from POM files in Hadoop and sub-projects

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: build
• Target Version/s: 2.8.0
• Hadoop Flags: Incompatible change, Reviewed

Release Note

Dependencies on commons-httpclient have been removed. Projects with undeclared transitive dependencies on commons-httpclient, previously provided via hadoop-common or hadoop-client, may find this to be an incompatible change. Such project are also potentially exposed to the commons-httpclient CVE, and should be fixed for that reason as well.

• Assignee: Matthew Foley Assign to me
• Reporter: Matthew Foley
• Created: 15/Jul/16 23:09
• Updated: 30/Aug/16 05:48
• Resolved: 21/Jul/16 23:09

Description

In branch-2.8 and later, the patches for various child and related bugs listed in HADOOP-10105, most recently including HADOOP-11613, HADOOP-12710, HADOOP-12711, HADOOP-12552, and HDFS-10623, eliminate all use of "commons-httpclient" from Hadoop and its sub-projects (except for hadoop-tools/hadoop-openstack; see HADOOP-11614).

However, after incorporating these patches, "commons-httpclient" is still listed as a dependency in these POM files:

• hadoop-project/pom.xml
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-registry/pom.xml

We wish to remove these, but since commons-httpclient is still used in many files in hadoop-tools/hadoop-openstack, we'll need to add the dependency to

• hadoop-tools/hadoop-openstack/pom.xml
  (We'll add a note to HADOOP-11614 to undo this when commons-httpclient is removed from hadoop-openstack.)
  In 2.8, this was mostly done by HADOOP-12552, but the version info formerly inherited from hadoop-project/pom.xml also needs to be added, so that is in the branch-2.8 version of the patch.

Other projects with undeclared transitive dependencies on commons-httpclient, previously provided via hadoop-common or hadoop-client, may find this to be an incompatible change. Of course that also means such project is exposed to the commons-httpclient CVE, and needs to be fixed for that reason as well.