Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-1885

Master caches DNS name resolution forever

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.16.0
    • master
    • None

    Description

      TSDescriptor::GetTSAdminProxy() and TSDescriptor::GetConsensusProxy() will return the same proxy instances over and over. Normally, this is a reasonable optimization. But suppose the IP address of the tserver changes (due to a DHCP lease expiring or some such). Now these methods will be returning unusable proxies, and there's no way to "reset" them.

      Admittedly this scenario is a little contrived: if a tserver's IP address suddenly changes, a bunch of other stuff will break too. The tserver will probably need to be restarted (since it's bound to a socket whose address no longer exists), and consensus may be thoroughly wrecked due to built-in host/port assumptions (see KUDU-418).

      An issue like this was reported by a user in Slack, who was running a master and tserver on the same box. The symptom was "half-open" communication between them: the tserver could heartbeat to the master, but the master could not send RPCs to the tserver.

      Attachments

        Issue Links

          Activity

            People

              awong Andrew Wong
              adar Adar Dembo
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: