[JAMES-3713] DLP matcher: enable caching

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: master
• Fix Version/s: 3.7.0
• Component/s: data, Mailet Contributions
• Labels: perf
• Assignee: Unassigned
• Reporter: Benoit Tellier
• Created: 16/Feb/22 05:02
• Updated: 17/Feb/22 06:49
• Resolved: 17/Feb/22 06:49

Description

Querying DLP rules on top of Cassandra relies on Event sourcing, involves reading a potentially large event stream and involves some SERIAL reads (LightWeight transactions) for each processed emails. Conducting flame graphs on Cassandra tend to show 20% of its activity consists of Paxos transactions for LWTs and in my setup DLP accounts for ~10% of the LWTs.

DLP rules depending on the domain of the sender, the cardinaility is extremely low.

Also, we can afford a window of inconsistencies in rules application (eg a minute)

This makes caching very appealing...

Issue Links

• links to: GitHub Pull Request #884