Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8981

Virtual IP address support

    XMLWordPrintableJSON

    Details

      Description

      I couldn't find support for virtual IP addresses in YARN framework. 

      This would be great if we have a docker-on-yarn service and if it for example has to be failed over to another physical host, clients can still find it. 

      So the idea is for YARN to bring up that virtual IP address (an additional/secondary IP address ) on a physical host where that particular docker container is running, so the clients that use that container's services don't have to change connection details every time that container moves around in YARN cluster.

      Similarly to virtual IP addresses in Kubernetes world:

      https://kubernetes.io/docs/concepts/services-networking/service/#virtual-ips-and-service-proxies

      One implementation could be through `ip address add` \ `ip address remove`.

      Kubernetes uses a more complicated `kube-proxy`, similarly to `docker-proxy` process in pure dockers / non-kubernetes docker deployments.

      Another approach is running a separate DNS service for a DNS subdomain (main DNS server would have to forward all requests for that DNS subdomain to a YARN DNS service). In Oracle Clusterware similar process is called GNS:

      https://docs.oracle.com/en/database/oracle/oracle-database/12.2/cwsol/about-the-grid-naming-service-vip-address.html#GUID-A4EE0CC6-A5F1-4507-82D6-D5C43E0F1584

      Would be great to have support for either virtual IP addresses managed by YARN directly or something similar to Oracle's GNS dns service.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Tagar Ruslan Dautkhanov
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated: