Description
Attempts to access HBase from Spark executors will fail at the auth to the metastore with: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
This is because HBase Kerberos auth token is not send to the executor. Will need to have something similar to obtainTokensForNamenodes(used for HDFS) in yarn/Client.scala. Storm also needed something similar: https://github.com/apache/storm/pull/226
I've created a patch for this that required an HBase dependency in the YARN module that we've been using successfully at eBay but am working on a version that does not require the HBase dependency by calling the class loader. Should be ready in a few days.
Attachments
Issue Links
- relates to
-
SPARK-7789 sql on security hbase:Token generation only allowed for Kerberos authenticated clients
- Resolved
-
SPARK-9833 Add options to explicitly disable delegation token retrieval for non-HDFS
- Resolved
- links to