Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
None
Description
For worker2worker communication, we use O(n^2) DNS lookups, where n is the number of workers. If network infrastructure is less reliable, combined with a large number of workers, these DNS lookups occasionally fail.
Fortunately, we don't always need DNS lookups, as in many clusters hosts can communicate with each other directly using IP addresses. Let's add a configuration option to use IP address instead of the host name.