[ARTEMIS-3745] OOM with PostgreSQL when loading large amount of messages from DB

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.19.1
• Component/s: Broker
• Assignee: Unassigned
• Reporter: Apache Dev
• Created: 25/Mar/22 17:42
• Updated: 14/Sep/22 19:08

Description

Scenario:

• Broker using PostgreSQL DB storage
• Address memory (about half of heap, with heap defined as 4GB) is filled with persistent messages
• Broker is restarted
• OutOfMemory occurs during broker startup when broker loads messages from DB

Issue occurs in:

org.apache.activemq.artemis.jdbc.store.journal.JDBCJournalImpl#load(org.apache.activemq.artemis.core.journal.LoaderCallback)

It depends on PostgreSQL JDBC driver default behavior, loading all result-set rows in memory.

In order to fetch smaller chunks of rows, it is possible to use this pattern:

connection.setAutoCommit(false);
selectJournalRecords.setFetchSize(FETCH_SIZE);

Reference (see "Getting results based on a cursor"): https://jdbc.postgresql.org/documentation/head/query.html