[OAK-6890] Background threads might not be automatically restarted

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.7.11, 1.8.0, 1.6.14
• Component/s: segment-tar
• Labels: resilience
• Assignee: Francesco Mari
• Reporter: Francesco Mari
• Created: 31/Oct/17 15:13
• Updated: 06/Sep/18 08:12
• Resolved: 07/Nov/17 10:01

Description

The background threads used in FileStore are implemented by wrapping Runnable instances in SafeRunnable, and by handing the SafeRunnable instances over to a ScheduledExecutorService.

The documentation of ScheduledExecutorService#scheduleAtFixedRate states that "if any execution of the task encounters an exception, subsequent executions are suppressed". But a SafeRunnable always re-throws any Throwable that it catches, effectively preventing itself from executing again in the future.

There is more than one solution to this problem. One of these is to never re-throw any exception. Even if it doesn't always make sense, e.g. in case of an OutOfMemoryError, never re-throwing an exception would better fulfil the assumption that background threads should always be up and running even in case of error.