[AMQ-7143] Temporary transaction file (PageFile) being opened and closed many times, causing poor performance on high latency FS

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 5.15.8
• Fix Version/s: 5.15.9, 5.16.0
• Component/s: KahaDB
• Patch Info: Patch Available
• Assignee: Christopher L. Shannon
• Reporter: Alan Protasio
• Created: 30/Jan/19 06:03
• Updated: 31/Jan/19 16:27
• Resolved: 31/Jan/19 16:11

Description

Hi,

This is an optimization when we have a transaction with many writes (bigger than 10mb by default) and activemq creates a temporary transaction file (pageFile transaction).

The problem is when this transaction is committed, this temporary file is opened and closed many times (number of writes inside the transaction), causing poor performance (this is operation "freezes the world" during the checkpoint).

 

See:

https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/page/PageFile.java#L1147

https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/page/PageFile.java#L203

 

Note that is the recoveryFile is enabled, we open and close the temporary 3 times for each index write.

There is also a small bug that if the transaction is rolledBack, the temporary file is left there forever, see:

https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/page/Transaction.java#L686

https://github.com/apache/activemq/blob/master/activemq-kahadb-store/src/main/java/org/apache/activemq/store/kahadb/disk/page/PageFile.java#L1200

 

As the transaction is rolledback, no writes will be performed and so, the file will never be deleted.

The proposed change, use the same "RandomAccessFile" object used in the transaction and close if when all writes is done (so we stop opening and closing it for each write, and only do it one time per transaction).

 

Thanks