Currently the Speculative Retry Policy takes a single string as a parameter, this can be NONE, ALWAYS, 99PERCENTILE (PERCENTILE), 50MS (CUSTOM).
The problem we have is when a single host goes into a bad state this drags up the percentiles. This means if we are set to use p99 alone, we might not speculate when we intended to to because the value at the specified percentile has gone so high.
As a fix we need to have support for something like min(99percentile,50ms)
this means if the normal p99 for the table is <50ms, we will still speculate at this value and not drag the happy path tail latencies up... but if the p99th goes above what we know we should never exceed we use that instead.