Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-22290

Starting second context in same JVM fails to get new Hive delegation token



    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.3.0
    • Fix Version/s: 2.3.0
    • Component/s: Spark Core
    • Labels:


      Consider the following pyspark script:

      sc = SparkContext()
      // do stuff
      // do some other stuff
      sc = SparkContext()

      That code didn't use to work at all in 2.2 (failure to create the second context), but makes more progress in 2.3. But it fails to create new Hive delegation tokens; you see this error in the output:

      17/10/16 16:26:50 INFO security.HadoopFSDelegationTokenProvider: getting token for: DFS[DFSClient[clientName=DFSClient_NONMAPREDUCE_-1714191595_19, ugi=blah(auth:KERBEROS)]]
      17/10/16 16:26:50 INFO hive.metastore: Trying to connect to metastore with URI blah
      17/10/16 16:26:50 INFO hive.metastore: Connected to metastore.
      17/10/16 16:26:50 ERROR metadata.Hive: MetaException(message:Delegation Token can be issued only with kerberos authentication. Current AuthenticationMethod: TOKEN)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result$get_delegation_token_resultStandardScheme.read(ThriftHiveMetastore.java)
              at org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$get_delegation_token_result.read(ThriftHiveMetastore

      The error is printed in the logs but it doesn't cause the app to fail (which might be considered wrong).

      The effect is that when that old delegation token expires the new app will fail.

      But the real issue here is that Spark shouldn't be mixing delegation tokens from different apps. It should try harder to isolate a set of delegation tokens to a single app submission.

      And, in the case of Hive, there are many situations where a delegation token isn't needed at all.


          Issue Links



              • Assignee:
                vanzin Marcelo Masiero Vanzin
                vanzin Marcelo Masiero Vanzin
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Created: