XML-RPC

Streaming mode not working as documented

Created: 27/Sep/07 10:53 AM   Updated: 10/May/09 05:36 PM
Return to search
Component/s: None
Affects Version/s: 3.1
Fix Version/s: None

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works streaming.patch 2008-01-16 03:57 PM Andreas Sahlbach 25 kB
Text File Licensed for inclusion in ASF works xmlrpc.patch 2007-09-27 11:23 AM Andreas Sahlbach 2 kB
Environment: Gentoo / Sun JDK 1.6
Issue Links:
Reference
 


 Description  « Hide
Here is my mail posted in the developer mailing list describing the issue(s):
Hi xmlrpc-gurus!

I am trying to migrate my projects from xmlrpc 2.0 to xmlrpc 3.1. I need to migrate one of the clients and the server, so I am very interested, that this part of the documentation is true:

> If streaming mode is disabled, then the server will always behave like a standard XML-RPC server. Otherwise, the server will verify, whether the client sends a content-length header. If so, then the server assumes that
> the client is able to accept a missing content-length header in the response as well. Otherwise, the server will still disable streaming for this particular requests. In other words, traditional clients will still receive a traditional > response and one server can serve both data types.

Unfortunately during verification of this I encountered two problems:

1) client: I am using the sun classes on a linux system. It looks like that it doesn't actually matters if I set contentLengthOptional and enabledForExtensions to tue or false. The request _always_ contains a content-length header. I debugged it but couldn't find place, where this header is added. I found the place in the client where the configuration was correctly read out and where the client was skipping the part to add this header. But nevertheless my request contains a content-length header at the end (I am using wireshark to sniff the network traffic). In the case I set the two configurations to true, the content-length header is always the last header in the header section. Can it be, that java is adding the content-length header by itself? If this is the case then using the content-length header for detection if the server should answer in streaming mode or not is not working!

2) server: I actually can't find the part in the sources where the server is honoring the content-length header in the request. It looks like the server is acting in streaming mode if I set both options to true and is not acting in streaming-mode, if I set both options to false. At least that is wireshark telling me. Could you give me a pointer to the code part that is doing the magic as stated in the documentation?

I don't want to nit-pick, but not becoming incompatible is essential for my service. Within the enterprise of my customer a number of clients are not under my control and I am in deep shit if they stop working :)

Thanks for your time guys!

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Andreas Sahlbach added a comment - 27/Sep/07 11:15 AM
To the client issue (1):

The header is probably added by java itself when using the sun classes as transport. Since java 1.5 it's possible to set an HttpURLConnection to streaming mode (see setChunkedStreamingMode () function), but XMLRPC must probably be compatible to 1.4, right? Therefore this is probably not an option.

When we are using commons-httpclient as transport, the code works as documented. So my suggestion to fix this issue on the client side is to update the documentation. Just clearly write, that for working streaming mode, the _client_ has to use commons-httpclient transport. (at least for my used jdk)

Andreas Sahlbach added a comment - 27/Sep/07 11:22 AM
To the server issue (2):

I've created a patch for XmlRpcServletServer that corrects the behavior, so that it's honoring the missing or not missing content-length header of the request. It now works as documented. See attached patch plz.

Andreas Sahlbach added a comment - 27/Sep/07 11:23 AM
Corrects the behavior therefore, that streaming mode is used only, if it is enabled on the server side AND if the client request does not contain a content-length header field.

Jochen Wiedmann added a comment - 27/Oct/07 06:14 PM
I have studied your patch. I agree that it ensures that the server behaves as documented.

However, judging from your observations on the clients behaviour, we currently have no reliable way to enforce that the server replies in streaming mode. This is bad.

I think that we should add an additional possibility to enforce the servers behaviour. For example, I could imagine that the client might add an attribute

    ex:contentLengthOptional="true"

with ex indicating the namespace http://ws.apache.org/xmlrpc/namespaces/extensions to the request document. Would you be ready to create a patch for that? I have added your patch, btw.

Andreas Sahlbach added a comment - 10/Jan/08 10:28 AM
I guess no one has worked on this in the mean time? No offence, I am just checking to prevent double work.

I have some time now and will try to create such a patch. I agree that this is the better solution. Especially because we encountered further problems with the existing "content-length" solution. In our more complex staging environments (QS and Productive) the solution does not work, because we are using an apache server as reverse proxy and the mod_rewrite package rejects the client posts with a "411 Content Length Required" in case of streaming mode requests.

So stay tuned for a patch in the next days.

PS: Happy new year and stuff to all of you :) ...

Andreas Sahlbach added a comment - 16/Jan/08 03:57 PM - edited
Please find attached the patch regarding the new streaming mode. The biggest difference to the old behavior is, that the client is not able to stream anymore and that the information, that the server should answer in streaming mode, now is part of the document (instead of the http header) Therefore contentLengthOptional=true needs enabledForExtensions=true.

I had to change several test cases to reflect these new facts and you might want to review the test suite, because it was designed with the different previous contentLengthOptional implementation in mind. And of course you should review my documentation changes. :-)

The server and client works fine here on our development server/client. I will deploy it the next days/weeks to the different stages but I expect no problems.

The patch is based on http://svn.apache.org/repos/asf/webservices/xmlrpc/trunk@610730

Jochen Wiedmann added a comment - 10/May/09 05:36 PM
Alan, apart from the fact that the combination of keepalive and streaming mode depends on the use of HTTP/1.1 and chunky mode, I do not see any relationship between this issue and XMLRPC-169.