Details
-
Improvement
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
None
-
Operability
-
Normal
-
All
-
None
-
Description
The hints system defines a time window, i.e. max_hint_window_in_ms, to store the hints.
It defines no limit on how much data can be kept during the time window. The hints can grow excessively and make the node running out of disk. In such scenario, the operators have to truncate the hints manually.
I'd propose that in addition to the conventional hints window, operators should be able to define the maximum hints size per host, i.e. max_hints_size_per_host_in_mb, to provide an another layer of protection. A node stops to store hints for the down node whenever it reaches to the time cap or the size cap. In order to not surprise the users, the config should be disabled by default. It should also be configurable via JMX.
