[HADOOP-15832] Upgrade BouncyCastle to 1.60

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0, 3.3.0
• Fix Version/s: 3.3.0, 3.2.1
• Target Version/s: 3.2.0, 3.3.0
• Hadoop Flags: Reviewed
• Assignee: Robert Kanter
• Reporter: Robert Kanter
• Created: 09/Oct/18 00:24
• Updated: 07/Sep/21 16:51
• Resolved: 10/Oct/18 01:18

Description

As part of my work on YARN-6586, I noticed that we're using a very old version of BouncyCastle:

<dependency>
   <groupId>org.bouncycastle</groupId>
   <artifactId>bcprov-jdk16</artifactId>
   <version>1.46</version>
   <scope>test</scope>
</dependency>

The *-jdk16 artifacts have been discontinued and are not recommended (see http://bouncy-castle.1462172.n4.nabble.com/Bouncycaslte-bcprov-jdk15-vs-bcprov-jdk16-td4656252.html).
In particular, the newest release, 1.46, is from 2011!
https://mvnrepository.com/artifact/org.bouncycastle/bcprov-jdk16

The currently maintained and recommended artifacts are *-jdk15on:
https://www.bouncycastle.org/latest_releases.html
They're currently on version 1.60, released only a few months ago.

We should update BouncyCastle to the *-jdk15on artifacts and the 1.60 release. It's currently a test-only artifact, so there should be no backwards-compatibility issues with updating this. It's also needed for YARN-6586, where we'll actually be shipping it.

Issue Links

• breaks: HADOOP-17324 Don't relocate org.bouncycastle in shaded client jars
• breaks: YARN-8919 Some tests fail due to NoClassDefFoundError for OperatorCreationException
• breaks: YARN-8899 TestCleanupAfterKIll is failing due to unsatisfied dependencies
• is depended upon by: YARN-6586 YARN to facilitate HTTPS in AM web server
• is related to: HADOOP-16015 Add bouncycastle jars to hadoop-aws as test dependencies
• is related to: HADOOP-15874 Add Bouncy Castle License
• is required by: YARN-8448 AM HTTPS Support for AM communication with RMWeb proxy
• is superceded by: HADOOP-17898 Upgrade BouncyCastle to 1.69