[HADOOP-13476] CredentialProviderFactory fails at class loading from libhdfs (JNI)

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.0, 2.9.0, 3.0.0-alpha2
• Fix Version/s: 2.9.0, 3.0.0-alpha1
• Component/s: fs/s3
• Target Version/s: 2.9.0, 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: Aaron Fabbri
• Reporter: Aaron Fabbri
• Created: 09/Aug/16 20:43
• Updated: 30/Aug/16 01:11
• Resolved: 10/Aug/16 16:59

Description

This bug was discovered when trying to run Impala (libhdfs.so) with s3a and Java KeyStore credentials. Because JNI threads have a different classloader (bootstrap), we fail to load JavaKeyStoreProvider.

15:11:42.658087 26310 jni-util.cc:166] java.util.ServiceConfigurationError: org.apache.hadoop.security.alias.CredentialProviderFactory: Provider
org.apache.hadoop.security.alias.JavaKeyStoreProvider$Factory not found
at java.util.ServiceLoader.fail(ServiceLoader.java:231)
at java.util.ServiceLoader.access$300(ServiceLoader.java:181)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:365)
at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
at org.apache.hadoop.security.alias.CredentialProviderFactory.getProviders(CredentialProviderFactory.java:57)
at org.apache.hadoop.conf.Configuration.getPasswordFromCredentialProviders(Configuration.java:1950)
at org.apache.hadoop.conf.Configuration.getPassword(Configuration.java:1930)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getAWSAccessKeys(S3AFileSystem.java:366)
at org.apache.hadoop.fs.s3a.S3AFileSystem.getAWSCredentialsProvider(S3AFileSystem.java:415)
at org.apache.hadoop.fs.s3a.S3AFileSystem.initialize(S3AFileSystem.java:176)

Issue Links

• is related to: HDFS-7075 hadoop-fuse-dfs fails because it cannot find JavaKeyStoreProvider$Factory