[CAMEL-10577] Jetty9 producer only supports payloads up hardcoded limit (2MB)

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.18.2, 2.19.0
• Component/s: camel-jetty
• Estimated Complexity: Unknown
• Assignee: Andrea Cosentino
• Reporter: Christoph Läubrich
• Created: 08/Dec/16 14:06
• Updated: 13/Jan/17 08:53
• Resolved: 12/Jan/17 07:16

Description

Camel-Jetty9 uses BufferingResponseListener without any arguments when retriving content from URLs.

This leads to the problem that you can retrive a maximum of 2MiB from a server, e.g. the following code

public void configure() throws Exception {

                from("timer://test?repeatCount=1")
                .log("Start download...")
                .to("jetty:http://mirror.netcologne.de/apache.org/camel/apache-camel/2.18.1/apache-camel-2.18.1.zip")
                .log("Download done!");

            }

leads to

org.apache.camel.CamelExchangeException: JettyClient failed cause by: Buffering capacity exceeded. Exchange[ID-laeubi4core-60072-1481204922807-0-2]. Caused by: [java.lang.IllegalArgumentException - Buffering capacity exceeded]
	at org.apache.camel.component.jetty9.JettyContentExchange9.doTaskCompleted(JettyContentExchange9.java:156)
	at org.apache.camel.component.jetty9.JettyContentExchange9$2.onComplete(JettyContentExchange9.java:222)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
	at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:185)
	at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:456)
	at org.eclipse.jetty.client.HttpReceiver.terminateResponse(HttpReceiver.java:437)
	at org.eclipse.jetty.client.HttpReceiver.responseHeaders(HttpReceiver.java:280)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.headerComplete(HttpReceiverOverHTTP.java:230)
	at org.eclipse.jetty.http.HttpParser.parseHeaders(HttpParser.java:1038)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1266)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.parse(HttpReceiverOverHTTP.java:158)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:119)
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:69)
	at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:90)
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:114)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
	at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
	at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
	at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Buffering capacity exceeded
	at org.eclipse.jetty.client.util.BufferingResponseListener.onHeaders(BufferingResponseListener.java:75)
	at org.eclipse.jetty.client.ResponseNotifier.notifyHeaders(ResponseNotifier.java:103)
	at org.eclipse.jetty.client.ResponseNotifier.notifyHeaders(ResponseNotifier.java:95)
	at org.eclipse.jetty.client.HttpReceiver.responseHeaders(HttpReceiver.java:259)
	... 16 more

The maximum size should at least be configurable (even though it is not recommended to do so since it seem to reserve the full buffer space on each request then).
It might be even better to use InputStreamResponseListener instead to place an input stream in the body.

Issue Links

• links to: GitHub Pull Request #1388