Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-12568

Provide an option to specify network interface used by Spark remote client [Spark Branch]

    XMLWordPrintableJSON

    Details

    • Target Version/s:
    • Release Note:
      Introduced configuration: hive.spark.client.rpc.server.address

      Description

      Spark client sends a pair of host name and port number to the remote driver so that the driver can connects back to HS2 where the user session is. Spark client has its own way determining the host name, and pick one network interface if the host happens to have multiple network interfaces. This can be problematic. For that, there is parameter, hive.spark.client.server.address, which user can pick an interface. Unfortunately, this interface isn't exposed.

      Instead of exposing this parameter, we can use the same logic as Hive in determining the host name. Therefore, the remote driver connecting to HS2 using the same network interface as a HS2 client would do.

      There might be a case where user may want the remote driver to use a different network. This is rare if at all. Thus, for now it should be sufficient to use the same network interface.

        Attachments

        1. HIVE-12568.0-spark.patch
          7 kB
          Xuefu Zhang
        2. HIVE-12568.1-spark.patch
          7 kB
          Xuefu Zhang
        3. HIVE-12568.2-spark.patch
          9 kB
          Xuefu Zhang
        4. HIVE-12568.2-spark.patch
          9 kB
          Xuefu Zhang

          Issue Links

            Activity

              People

              • Assignee:
                xuefuz Xuefu Zhang
                Reporter:
                xuefuz Xuefu Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: