Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
While working on the deletedMessageVault, we targeted in MAILBOX-381 a first implementation on top of MailRepositories
That decision was unfortunate as then one can browse the content of the deletedMessageVault vault through webAdmin.
Thus we decided to implement DeletedMessageVault as a separate component.
That component has the following considerations:
- Should have a low storage cost
- Cold storage and delays upon reads are acceptable as one does not expect a restore/export to be fast
- Find an efficient way to implement retention (deleting too old emails) without a full scan
We decided to provide an implementation above the blobStore and a metaData store (cassandra)
Retention will be implemented with object storage buckets (one month = 1 bucket). We could then simply delete a bucket when all the mails it contains are older than the retention period.
Querying the deleteMessageVault will first be done with a full scan. Later enhancement can be done by providing a per-bucket (immutable) Lucene index - once the bucket is no longer the latest one.
Attachments
1.
|
Adding bucket capability into BlobStore | Closed | Unassigned | ||
2.
|
DeletedMessageMetadataVault API + contract +memory | Closed | Unassigned | ||
3.
|
DeletedMessage serialization & deserialization | Resolved | Unassigned | ||
4.
|
DeletedMessageVault implementation of append & search (full scan version) on top of BlobStore | Closed | Unassigned | ||
5.
|
DeletedMessageMetadataVault : Cassandra | Closed | Unassigned | ||
6.
|
DeletedMessageVault implemetation of retention | Closed | Unassigned | ||
7.
|
Integration tests for new DeletedMessageVault implementation | Closed | Unassigned |
Activity
Field | Original Value | New Value |
---|---|---|
Description |
While working on the deletedMessageVault, we targeted in That decision was unfortunate as then one can browse the content of the deletedMessageVault vault through webAdmin. Thus we decided to implement DeletedMessageVault as a separate component. That component has the following considerations: - Should have a low storage cost - Cold storage and delays upon reads are acceptable as one does not expect a restore/export to be fast - Find an efficient way to implement retention (deleting too old emails) without a full scan We decided to provide an implementation above the blobStore and a metaData store (cassandra) Retention will be implemented with object storage buckets (one month = 1 bucket). We could then simply delete a bucket when all the mails it contains are older than the retention period. Querying the deleteMessageVault will first be done with a full scan. Later enhancement can be done by providing a per-bucket (immutable) Lucene index - once the bucket is no longer the latest one. |
Fix Version/s | 3.4.0 [ 12344981 ] | |
Resolution | Fixed [ 1 ] | |
Status | Open [ 1 ] | Resolved [ 5 ] |
Status | Resolved [ 5 ] | Closed [ 6 ] |
I created the tasks for the tasks we plan to tackle in the very near future.
Note that to complete this work we will also need: