Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-16071

HoS RPCServer misuses the timeout in its RPC handshake

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: Spark
    • Labels:
      None

      Description

      Based on its property description in HiveConf and the comments in HIVE-12650 (https://issues.apache.org/jira/browse/HIVE-12650?focusedCommentId=15128979&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15128979), hive.spark.client.connect.timeout is the timeout when the spark remote driver makes a socket connection (channel) to RPC server. But currently it is also used by the remote driver for RPC client/server handshaking, which is not right. Instead, hive.spark.client.server.connect.timeout should be used and it has already been used by the RPCServer in the handshaking.
      The error like following is usually caused by this issue, since the default hive.spark.client.connect.timeout value (1000ms) used by remote driver for handshaking is a little too short.

      17/02/20 08:46:08 ERROR yarn.ApplicationMaster: User class threw exception: java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
      java.util.concurrent.ExecutionException: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
              at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:37)
              at org.apache.hive.spark.client.RemoteDriver.<init>(RemoteDriver.java:156)
              at org.apache.hive.spark.client.RemoteDriver.main(RemoteDriver.java:556)
              at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
              at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.lang.reflect.Method.invoke(Method.java:606)
              at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:542)
      Caused by: javax.security.sasl.SaslException: Client closed before SASL negotiation finished.
              at org.apache.hive.spark.client.rpc.Rpc$SaslClientHandler.dispose(Rpc.java:453)
              at org.apache.hive.spark.client.rpc.SaslHandler.channelInactive(SaslHandler.java:90)
      

        Attachments

        1. HIVE-16071.patch
          0.7 kB
          Chaoyu Tang
        2. HIVE-16071.patch
          0.7 kB
          Chaoyu Tang
        3. HIVE-16071.patch
          2 kB
          Chaoyu Tang

          Issue Links

            Activity

              People

              • Assignee:
                ctang.ma Chaoyu Tang
                Reporter:
                ctang.ma Chaoyu Tang
              • Votes:
                0 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: