Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-18627

Cannot connect to Hive metastore in client mode with proxy user

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Cannot Reproduce
    • Affects Version/s: 2.0.0
    • Fix Version/s: None
    • Component/s: YARN
    • Labels:
      None

      Description

      Marking as "minor" since the security story for client mode with proxy users is a little sketchy to start with, but it shouldn't fail, at least not in this manner. Error you get is:

      Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: GSS initiate failed
              at org.apache.thrift.transport.TSaslTransport.sendAndThrowMessage(TSaslTransport.java:232)
              at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:316)
              at org.apache.thrift.transport.TSaslClientTransport.open(TSaslClientTransport.java:37)
              at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:52)
              at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport$1.run(TUGIAssumingTransport.java:49)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1796)
              at org.apache.hadoop.hive.thrift.client.TUGIAssumingTransport.open(TUGIAssumingTransport.java:49)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:430)
              at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:240)
              at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
              at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1528)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:67)
              at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:82)
              at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3238)
              at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3257)
              at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3482)
              at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:225)
              at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:209)
              at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:332)
              at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:293)
              at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:268)
              at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:529)
              at org.apache.spark.sql.hive.client.ClientWrapper.<init>(ClientWrapper.scala:204)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
              at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:249)
      

      Cluster mode works fine.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                vanzin Marcelo Vanzin
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: