[SPARK-11570] ambiguous hostname resolving during startup

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 1.4.0, 1.4.1, 1.5.0, 1.5.1
• Component/s: Spark Core
• Labels: bulk-closed

Environment

standalone

• Assignee: Unassigned
• Reporter: Sergey Soldatov
• Created: 08/Nov/15 09:13
• Updated: 21/May/19 04:32
• Resolved: 21/May/19 04:32

Description

when master is running in standalone mode, it expects that the hostname will be provided by --host. This is done by the start-master.sh script which sets --ip to `hostname` (doesn't it looks weird?).
If someone running master directly (without start-master.sh), the hostname will be initialized as var host = Utils.localHostName(). Before SPARK-6440 that was exactly the host name where master was started. But now it returns ip address instead. That would lead to worker connectivity problems (like you may observer in BIGTOP-2113). If ip addresses are prohibited, than var host should not be initialized in that way.
Possible solutions - return the logic to set it to the host name or fail if no --host argument was provided.