[CAMEL-7521] Provide an option for unsynchronized aggregation when splitter is streaming and not parallel

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.13.2
• Fix Version/s: 2.14.0
• Component/s: camel-core
• Patch Info: Patch Available
• Estimated Complexity: Unknown
• Assignee: Claus Ibsen
• Reporter: Jerry Williamson
• Created: 18/Jun/14 15:55
• Updated: 04/Jul/14 11:31
• Resolved: 04/Jul/14 11:31

Description

We use a splitter route that splits very large files.
The splits need to be aggregated in order.
The splitter is therefore configured with streaming=true and parallelProcessing=false.

When there are multiple in-flight exchanges on the route (i.e. different files are being processed) the end-to-end processing time for each file is significantly impacted because each aggregation call is synchronized in MulticastProcessor.doAggregate method..

It is not uncommon, in our environment, to have thousands of splits per file and the synchronization is significantly impacting the throughput of the route.

I propose that an option be added (to splitter at least) to allow for an unsynchronized aggregation. The option would default to false for backward compatibility and would possibly be ignored if parallelProcessing = true.

The body of the MulticastProcess.doAggregate method could be moved to a new, unsynchronized method (say doAggregateInternal). The MulticastProcessor.doAggregate method could remain synchronized (again for backward compatibility) but the body of that method would then call the new unsynchronized doAggregateInternal method. In this way, existing code would remain synchronized.

The splitter, then, could call the unsynchronized doAggregateInternal method when the unsynchronizedAggregation option was true.