[CAMEL-6899] Stream component has various problems.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.12.1
• Fix Version/s: 2.11.3, 2.12.2, 2.13.0
• Component/s: camel-stream

Environment

Java 7, Mac OS X

• Patch Info: Patch Available
• Estimated Complexity: Novice
• Assignee: Claus Ibsen
• Reporter: Paolo Morandini
• Created: 25/Oct/13 08:31
• Updated: 12/Nov/13 10:11
• Resolved: 12/Nov/13 10:11

Description

Camel stream component has several issues (when using it to transfer big, finite, files):

• If using groupLines, this component can transfer only a number of lines that's a multiple of this parameter (ie. if groupLines=7 and a file consists of 20 lines you'll send two Exchanges containing rows 1-7 and 8-14, losing rows 15-20)

• there's no way to know when a stream finished, that renders the onCompletion() hook quite useless

• Stream producer can produce only from String or byte[], but stream consumer produces either String or List<String>, this makes impossible to use grouping to transfer files between two stream endpoints

Fastest workaround I found out is to add the chance to specify additional URL parameters in Endpoint configuration (i.e. stream:url?binary=true... ) thus allowing the Consumer to read chunks of files into byte[] and packing them into an Exchange.

My feeling is that the entire component could be enlisted to go under a deeper review and major refactoring.

I'm attaching a Patch for your evaluation that should temporarily fix this mis-behaviors, at leas in my working scenario (but I think they're pretty generic).