[ARTEMIS-2746] Reduce memory footprint of journal loading

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Francesco Nigro
• Created: 30/Apr/20 17:49
• Updated: 30/Apr/20 20:56

Description

While loading the journal, before reloading the messages into each queue, we store the reference to the message and some additional information (scheduledDeliveryTime and deliveryCount) in a LinkedHashMap per-queue.

These information are using a AddMessageRecord class that can weigth 28 bytes:
12 object header + 4 for Message + 8 for scheduledDeliveryTime + 4 deliveryCount + 4 padding on a JDK 64 bits with COOPS.

Often both scheduledDeliveryTime and deliveryCount are not used, hence we could save 28 bytes per message by using a separate data-structure to hold such additional records.
Such records could be stored in a separate LongObjectHasMap that would cost approximately (it depends on the load factor) per-entry:
8 (for the messageID key) + 4 (for the additional record reference) + 16 (for the record instance) = 28 bytes

In the worst case, given that the additional informations require the same amount of memory (28 bytes), the only downside would be some lookup cost while walking/updating both the data-structures, but in the best case (and most common cases) it means eg

100_000 elements = saved ~2.8 MB
1_000_000 elements = saved ~28 MB
10_000_000 elements = saved ~280 MB