[HADOOP-16810] Increase entropy to improve cryptographic randomness on precommit Linux VMs

• Type: Bug
• Status: In Progress
• Priority: Blocker
• Resolution: Unresolved
• Labels: pull-request-available
• Assignee: Ahmed Hussein
• Reporter: Ahmed Hussein
• Created: 17/Jan/20 15:05
• Updated: 22/Feb/21 14:48

Description

I was investigating a JUnit test (MAPREDUCE-7079 :TestMRIntermediateDataEncryption is failing in precommit builds) that was consistently hanging on Linux VMs and failing Mapreduce pre-builds.
I found that the test hangs slows or hangs indefinitely whenever Java reads the random file.

I explored two different ways to get that test case to work properly on my local Linux VM running rel7:

1. To install "haveged" and "rng-tools" on the virtual machine running Rel7. Then, start rngd service sudo service rngd start . This will fix the problem for all the components on the image including java, native and any other component.
2. Change java configuration to load urandom

   sudo vim $JAVA_HOME/jre/lib/security/java.security
   ## Change the line “securerandom.source=file:/dev/random” to read: securerandom.source=file:/dev/./urandom
   

The first solution is better because this will fix the problem for everything that requires SSL/TLS or other services that depend upon encryption.

Since the precommit build runs on Docker, then it would be best to mount /dev/urandom from the host as /dev/random into the container:

docker run -v /dev/urandom:/dev/random

For Yetus, we need to add the mount to the DOCKER_EXTRAARGS as follows:

DOCKER_EXTRAARGS+=("-v" "/dev/urandom:/dev/random")

...

Issue Links

• is required by: MAPREDUCE-7079 JobHistory#ServiceStop implementation is incorrect
• links to: GitHub Pull Request #2697