[HADOOP-15473] Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.6, 3.0.2
• Fix Version/s: 2.10.0, 3.2.0, 3.1.1, 2.9.2, 3.0.3, 2.7.7, 2.8.5
• Component/s: kms

Environment

JDK 8u171

• Hadoop Flags: Reviewed
• Assignee: Gabor Bota
• Reporter: Gabor Bota
• Created: 24/Apr/18 12:41
• Updated: 11/Jan/19 03:44
• Resolved: 25/May/18 16:47

Description

There is a new feature in JDK 8u171 called Enhanced KeyStore Mechanisms (http://www.oracle.com/technetwork/java/javase/8u171-relnotes-4308888.html#JDK-8189997).
This is the cause of the following errors in the TestKeyProviderFactory:

Caused by: java.security.UnrecoverableKeyException: Rejected by the jceks.key.serialFilter or jdk.serialFilter property
	at com.sun.crypto.provider.KeyProtector.unseal(KeyProtector.java:352)
	at com.sun.crypto.provider.JceKeyStore.engineGetKey(JceKeyStore.java:136)
	at java.security.KeyStore.getKey(KeyStore.java:1023)
	at org.apache.hadoop.crypto.key.JavaKeyStoreProvider.getMetadata(JavaKeyStoreProvider.java:410)
	... 28 more

This issue causes errors and failures in hbase tests right now (using hdfs) and could affect other products running on this new Java version.

Issue Links

• breaks: HBASE-20535 Check the UT TestSaslFanOutOneBlockAsyncDFSOutput which is always flaky
• is depended upon by: HADOOP-15509 Release Hadoop 2.7.7
• is duplicated by: HADOOP-15445 TestCryptoAdminCLI test failure when upgrading to JDK8 patch 171.
• is duplicated by: HADOOP-15519 KMS fails to read the existing key metadata after upgrading to JDK 1.8u171
• is related to: HBASE-20538 Upgrade our hadoop versions to 2.7.7 and 3.0.3
• mentioned in: Page Loading...