[IGNITE-9679] Document critical workers liveness checking implementation

• Type: Task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7
• Component/s: documentation
• Assignee: Denis A. Magda
• Reporter: Andrey Kuznetsov
• Created: 24/Sep/18 18:43
• Updated: 21/Nov/18 23:02
• Resolved: 21/Nov/18 23:02

Description

Newly implemented critical worker thread liveness checks should be mentioned in Ignite Documentation. Brief description of the functionality follows.

Ignite node has a number of critical worker threads that should be alive and responsive, otherwise node's health is not guaranteed. These threads monitor each other periodically and track two aspects for a thread being checked:

• whether it's alive;
• whether it updates its internal heartbeat timestamp.
  Whenever at least one of the above conditions is violated, checker thread logs the error and calls currently configured FailureHandler.

IgniteConfiguration.SystemWorkerBlockedTimeout configuration property affects monitoring behavior. At runtime monitoring settings can be changed via FailureHandlingMxBean.

By default, liveness checks are enabled, but blocked system worker detection will not lead to failure handler invocation, see FailureProcessor#getDefaultFailureHandler .

Issue Links

• is blocked by: IGNITE-9737 Ignite WatchDog service should be configurable
• is related to: IGNITE-6587 Ignite watchdog service