[SPARK-20529] Worker should not use the received Master address

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.3, 2.0.2, 2.1.0, 2.2.0
• Fix Version/s: 2.2.0
• Component/s: Spark Core
• Assignee: Shixiong Zhu
• Reporter: Shixiong Zhu
• Created: 28/Apr/17 23:16
• Updated: 16/May/17 17:38
• Resolved: 16/May/17 17:38

Description

Right now when worker connects to master, master will send its address to the worker. Then worker will save this address and use it to reconnect in case of failure.

However, sometimes, this address is not correct. If there is a proxy between master and worker, the address master sent is not the address of proxy.

Issue Links

• is duplicated by: SPARK-20531 Spark master shouldn't send its address back to the workers over proxied connections
• links to: [Github] Pull Request #17821 (zsxwing)