[SPARK-23759] Unable to bind Spark UI to specific host name / IP

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.2.0
• Fix Version/s: 2.2.2, 2.3.1, 2.4.0
• Component/s: Spark Core, Web UI
• Assignee: Felix Albani
• Reporter: Felix Albani
• Created: 20/Mar/18 22:33
• Updated: 23/Mar/18 17:38
• Resolved: 23/Mar/18 17:37

Description

Ideally, exporting SPARK_LOCAL_IP=<private-interface-ip-address> in spark2 environment should allow Spark2 History server to bind to private interface however this is not working in spark 2.2.0

 

Spark2 history server still listens on 0.0.0.0

[root@sparknode1 ~]# netstat -tulapn|grep 18081
tcp        0      0 0.0.0.0:18081               0.0.0.0:*                   LISTEN      21313/java
tcp        0      0 172.26.104.151:39126        172.26.104.151:18081        TIME_WAIT   -

On earlier versions this change was working fine:

[root@dwphive1 ~]# netstat -tulapn|grep 18081
tcp        0      0 172.26.113.55:18081         0.0.0.0:*                   LISTEN      2565/java

 

This issue not only affects SHS but also Spark UI in general

 

Issue Links

• links to: [Github] Pull Request #20867 (felixalbani)
• links to: [Github] Pull Request #20883 (felixalbani)