[#AMQ-1689] (NIO)DataFileAppender does to many sync or force calls when there is little or no concurrency - Apache ActiveMQ, Apache ServiceMix, Apache Camel

ActiveMQ

(NIO)DataFileAppender does to many sync or force calls when there is little or no concurrency

Created: 25/Apr/08 06:13 AM Updated: 01/May/08 02:18 AM

Component/s:

Message Store

Affects Version/s:

5.1.0

Fix Version/s:

5.1.0

Time Tracking:

Not Specified

File Attachments:

	File Name	Date Attached	Attached By	Size
	fileAppenderBatch.patch	2008-04-25 06:17 AM	Gary Tully	27 kB

Environment:

Windows, NIO force and File sync perform poorly

Patch Info:

Patch Available

Description

« Hide

Where NIO channel force and fileDescriptor.sync perform poorly and there is little concurrently, the kaha DataFileAppender performs poorly.
A test case where a single producer and consumer operate from either ends of a networked pair of brokers shows the problem. It takes ~3 minutes to send 1000 messages on windows!. The problem is that 70% of the time is spent in NIO channel.flush()

The flush is batched if there are concurrent writes, but otherwise, a single write is flushed immediately.
My proposal is to have the batch processor batch till a batch is full or till a sync flag is present in a write command or till some timeout expires or if the appender is closed.
This reduces the number of force calls and with this change, my test case can do 1000 messages in ~1minute.

All

Comments

Work Log

Change History

Subversion Commits

FishEye

Crucible

Sort Order:

Gary Tully made changes - 25/Apr/08 06:17 AM

Field	Original Value	New Value
Attachment		fileAppenderBatch.patch [ 16386 ]

Rob Davies made changes - 29/Apr/08 03:43 AM

Assignee

Hiram Chirino [ chirino ]

Hiram Chirino made changes - 29/Apr/08 02:01 PM

Resolution		Fixed [ 1 ]
Fix Version/s		5.1.0 [ 11802 ]
Status	Open [ 1 ]	Resolved [ 5 ]

Gary Tully made changes - 01/May/08 02:18 AM

Status

Resolved [ 5 ]

Closed [ 6 ]