[CASSANDRA-14865] Cascading calls to read retries, system_auth, and read repairs

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Not A Problem
• Fix Version/s: 3.11.1
• Component/s: Feature/Authorization
• Severity: Normal
• Assignee: Unassigned
• Reporter: Pedro Gordo
• Created: 02/Nov/18 11:03
• Updated: 16/Apr/19 09:29
• Resolved: 26/Mar/19 23:03

Description

Roles validity and permission cache values are the default ones. Same thing for the read-repair chance.

We have a cluster with 3 data centers. We have noticed that in 2 of the data centers (NEC and CN) we have multiple calls to speculative read retries (rapid read protection), roles (instead of using cached values within the same tracing session), and multiple read repair messages.

We would expect calls to roles, within sequential read sessions, because the cache could be turned, but not so many calls within the same tracing session. Same thing for read-retries, and read repair messages. We are discarding blocking read repairs possibility because we consistently get the same results when doing the query several times.

It feels like something is cascading calls to these mechanisms regardless of conditions that would prevent them from being called (cached roles values for instance).

I have attached tracing files from the 3 data centers. Please let me know if more info is needed.