[SPARK-12267] Standalone master keeps references to disassociated workers until they sent no heartbeats

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 1.6.0
• Component/s: Spark Core
• Assignee: Shixiong Zhu
• Reporter: Jacek Laskowski
• Created: 10/Dec/15 19:21
• Updated: 16/Dec/15 00:09
• Resolved: 13/Dec/15 06:01

Description

While toying with Spark Standalone I've noticed the following messages
in the logs of the master:

INFO Master: Registering worker 192.168.1.6:59919 with 2 cores, 2.0 GB RAM
INFO Master: localhost:59920 got disassociated, removing it.
...
WARN Master: Removing worker-20151210090708-192.168.1.6-59919 because
we got no heartbeat in 60 seconds
INFO Master: Removing worker worker-20151210090708-192.168.1.6-59919
on 192.168.1.6:59919

Why does the message "WARN Master: Removing
worker-20151210090708-192.168.1.6-59919 because we got no heartbeat in
60 seconds" appear when the worker should've been removed already (as
pointed out in "INFO Master: localhost:59920 got disassociated,
removing it.")?

Could it be that the ids are different - 192.168.1.6:59919 vs localhost:59920?

I started master using ./sbin/start-master.sh -h localhost and the
workers ./sbin/start-slave.sh spark://localhost:7077.

Issue Links

• links to: [Github] Pull Request #10261 (zsxwing)