[SPARK-12411] Reconsider executor heartbeats rpc timeout - ASF JIRA

Log work

Agile Board

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Bulk Copy Attachments

Bulk Move Attachments

Voters

Watch issue

Watchers

Create sub-task

Convert to sub-task

Move

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

Delete

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 1.6.1, 2.0.0
Component/s: Spark Core
Labels:
None

Target Version/s:

1.6.1, 2.0.0

Description

Currently, the timeout for checking when an executor is failed is the same as the timeout of the sender ("spark.network.timeout") which defaults to 120s.

This means if there is a network issue, the executor only gets one try to heartbeat which probably causes the failure detection to be flaky.

The executor has a config to control how often to heartbeat (spark.executor.heartbeatInterval) which defaults to 10s. This combination of configs doesn't seem to make sense. The heartbeat rpc timeout should probably be less than or equal to the heartbeatInterval.