Hi, Sometimes a web based application expects to receive a request with content type multipart/form-data and does not accept a simple post request. A browser creates a multipart/form-data request even if no file is uploaded, if the user leaves the upload-field empty. However, with jmeter it is not possible to create a multipart/form-data request without uploading a file, jmeter just sends a simple post form. It would be nice if there was a checkbox in the GUI to create a multipart/form-data request even without uploading a file. Also, while recording a workflow with a multipart/form-data request without uploaded file with the jmeter proxy, the proxy resends it as a simple request. It would be nice if the content-type of the original request would be preserved in the call by the proxy. Even more problematic is that the HTTP header still proclaims the sent request to have the content type multipart/form-data, even if the body does not contain the multipart boundary. Then, if a file is choosen which is not present in the directory where jmeter was started, the Jmeter proxy displays a FileNotFoundException in the Browser, although the proxy is located on the same host as the browser and the full path was given when selecting the file. To prevent this error, I had to put a file with the same name in the directory where jmeter was started from. If possible, it would be nice if the jmeter proxy would use the full path when re-sending the request to the server. (I don't know enough about proxies: probably the image is already contained in the request sent by the browser, so it is not necessary to access the file system again ???) Cheers, Thomas
I notice a problem in org.apache.jmeter.protocol.http.sampler.PostWriter at line 137 in setHeaders.(line 98 at version 2.0.1) The filename should be acquired from getFilename(). The filename should look like : String filename = sampler.getFilename(); and not : String filename = sampler.getFileField(); Thanks, Takuya
Created attachment 19792 [details] Suggested patch This is a sugggested patch for improving the HTTP POST request functionality. This patch adds support for using multipart/form-data even if no file is being uploaded. I have not currently added a property in the jmeter.properties file to control that you want multipart/form-data. In the start, I want to let the Proxy Server create "HTTP Samplers" that are using multipart/form-data. More headers are set both on the request, and in the multiparts, for both HTTPSampler and HTTPSampler2. That means that both content-type, and content-length are now set. Also the encoding used are set for the multiparts. I think this means that different encodings are now supported for POSTs. You have to set the "content encoding" in the "HTTP Request" gui, to have the POST sent with the correct encoding. This applies for both x-www-form-urlencoded and multipart/form-data. The "HTTPSamplerBase.getQyeryString" could not be used, since that always encodes to UTF-8, also the parameter values. Both the HTTPSampler and HTTPSampler2 is changed, so that the "querystring" which is set on the created sample, more fully represents what is being sent to the web server. For multipart/form-data, that means that the multiparts are displayed, except the actual file content. It changes PostWriter, by changing methods to non-static, i.e. requiring that PostWriter is instantiated. This is needed to keep state set in "setHeaders", and reuse that state when doing "sendPostHeaders". What I currently do not like about the patch : HTTPSampler and HTTPSampler2 are using different "default encoding" if none is specified in the "HTTP Request" GUI. HTTPSampler uses iso-8859-1, while HTTPSampler2 uses us-ascii. I think both should use the same default encoding. I think the default should be iso-8859-1, I think that is what most browsers use by default. The HTTPSampler currently provides the full path to the file in the header of the file multipart. I think it is correct to only include the filename. That is also what the HTTPSampler2 does. I am not 100% confident that I am handling the "httpargument.isalwaysencoded" correctly. As of now, I ignore it, and always encode the parameter name and value. I think the "isalwaysencoded" only applies to parameters that should be used in a HTTP GET request. But I might very well be wrong. Conclusion : I think this patch has potential for solving a number of issues related to HTTP POST handling. But please have a review of the code, and ask and comment about the code. I'll also look into making some automated tests for HTTP POST handling.
Created attachment 19814 [details] Patch for unit test for svn version of PostWriter This adds unit tests for the PostWriter that is currently in SVN. The code lines with // at the very start of the line, will fail in this version, if uncommented. The whole "testSendFormData_Multipart" will also fail, since there is no way to tell HTTPSampler to write a multipart/form-data request, if you are not uploading files. The "testSendPostData_FileAsBody" will also fail for the last test, because the current PostWriter writes the file content to the body, if both a filename and some form data are present in the HTTPSampler.
Created attachment 19815 [details] Unit test for the suggested updated PostWriter / HTTPSampler in patch above This is unit tests for the suggested updated PostWriter / HTTPSampler, the patch for that is in this same bugzilla issue. This unit tests passes all tests. So it shows that the content length is set correctly for all requests. And it shows that multipart/form-data works even if you have no files to upload. And it shows that posting of UTF-8 values as part of multipart/form-data works, which is not working in the current PostWriter. The lines 113 and 114 in the unit test for the current PostWriter looks like this (i.e. look in the patch for unit test for current PostWriter) // checkContentLength(connection, expectedFormBody.length); // checkArraysHaveSameContent(expectedFormBody, connection.getOutputStreamContent()); because it fails for current PostWriter. And it shows that if a HTTPSampler contains both form parameters, and just a filename, then only the form parameters will be sent, and not the file as post body.
Created attachment 19826 [details] Patch for unit test for current svn version of PostWriter Patch for unit test for svn version of PostWriter. This adds on the patch for current svn version of PostWriter, by adding test cases for adding HTTPArguments with values which have been urlencoded by hand adding it to the HTTPSampler. This patch also contains a new class TestHTTPSamplersAgainstHttpMirrorServer, which executes real samples against the HttpMirrorServer. The test does not care about what reply one gets from the HttpMirrorServer, but it allows to test on the request headers and content created by the HTTPSampler and HTTPSampler2. As before, the lines starting with // at the very beginning of the lines, are either tests which fail with current svn versions of the HTTPSampler/2 code, or that depends on new code being in place.
Created attachment 19827 [details] Updated patch for unit testing for new suggested PostWriter/HTTPSampler/2 This patch contains updated PostWriterTest.java for the suggested new PostWriter and HTTPSampler and HTTPSampler2, i.e. the "main" suggested patch of this bugzilla entry. It also contains the TestHTTPSamplersAgainstHttpMirrorServer, where all tests passes when the suggested patch for HTTPSampler, PostWriter and HTTPSampler2 is in place.
Created attachment 19846 [details] Updated patch for unit testing of current svn version of PostWriter/HTTPSampler/2 Updated, and I think final, version of unit testing of PostWriter, HTTPSampler and HTTPSampler2. Compared to the previous suggested patch for unit testing og current svn versions of PostWriter, HTTPSampler and HTTPSampler2, this patch adds checking against the reply sent from the mirror server. The HTTP Mirror server is sending back the request headers and body we sent to the mirror server. I have also added a test on using JMeterVariables, to replace values in form data., i.e. use variables as part of a form parameter value.
Created attachment 19847 [details] Updated patch for unit testing for new suggested PostWriter/HTTPSampler/2 Updated, and I think final, version of unit testing of PostWriter, HTTPSampler and HTTPSampler2. Compared to the previous suggested patch for unit testing of the suggested new versions of PostWriter, HTTPSampler and HTTPSampler2, this patch adds checking against the reply sent from the mirror server. The HTTP Mirror server is sending back the request headers and body we sent to the mirror server. I have also added a test on using JMeterVariables, to replace values in form data., i.e. use variables as part of a form parameter value.
(In reply to comment #7) > Created an attachment (id=19846) [edit] > Updated patch for unit testing of current svn version of > PostWriter/HTTPSampler/2 > Updated, and I think final, version of unit testing of PostWriter, HTTPSampler > and HTTPSampler2. > Compared to the previous suggested patch for unit testing og current svn > versions of PostWriter, HTTPSampler and HTTPSampler2, this patch adds checking > against the reply sent from the mirror server. The HTTP Mirror server is > sending back the request headers and body we sent to the mirror server. > I have also added a test on using JMeterVariables, to replace values in form > data., i.e. use variables as part of a form parameter value. When the above patch is applied, I get 4 test failures: checkContentTypeUrlEncoded(PostWriterTest.java:821) checkContentTypeMultipart(PostWriterTest.java:817) checkContentLength(PostWriterTest.java:825) checkPostRequestFormMultipart(TestHTTPSamplersAgainstHttpMirrorServer.java:295) Are these expected? The code does not indicate so.
Created attachment 19859 [details] Updated patch for unit testing of current svn version of PostWriter/HTTPSampler/2 In this updated patch for unit testing of current svn version of PostWriter/HTTPSampler/2, I have commented out all tests that were failing expectedly. Since PostWriter/HTTPSampler/2 does not produce multipart/form-data requests, if only form parameters are present, the whole method for testing that has been commented out. I have also commented out tests in PostWriter for testing sending parameter which has been url decoded beforehand, and when the request is made using a content type different from UTF-8, because that is not working in the current PostWriter version. So if you now apply this patch, all the unit tests should pass.
Applied patch 19859; SVN r524624.
Patch 19792 applied to SVN (r524696). This required some fixes to the test cases for the additional headers.
Created attachment 19892 [details] Fix bug in PostWriter2 for content transfer encoding, and add unit tests Attached is patch for PostWriter, to fix bug where "Content-Transfer-Encoding: 8bit" was sent instead of the correct "Content-Transfer-Encoding: binary" for the file multipart. This makes PostWriter work like HTTPSampler2. I have also enabled more of the unit tests in TestHTTPSamplersAgainstHttpMirrorServer, which were commented out. And I have also added unit tests for file upload in that class, although I am not doing as much testing on the post body sent as I wish I had. I have also changed the ENCODING of PostWriter to "ISO-8859-1", i.e. uppercase. I do not think it is case sensitive, but I think it is most common to use uppercase. See for example : http://www.iana.org/assignments/character-sets which says "Alias: ISO-8859-1 (preferred MIME name)" I have also made a constant HTTP_ENCODING for both PostWriterTest and TestHTTPSamplersAgainstHttpMirrorServer.
(In reply to comment #13) > Created an attachment (id=19892) [edit] > Fix bug in PostWriter2 for content transfer encoding, and add unit tests > Attached is patch for PostWriter, to fix bug where "Content-Transfer- Encoding: > 8bit" was sent instead of the correct "Content-Transfer-Encoding: binary" for > the file multipart. This makes PostWriter work like HTTPSampler2. > I have also enabled more of the unit tests in > TestHTTPSamplersAgainstHttpMirrorServer, which were commented out. And I have > also added unit tests for file upload in that class, although I am not doing as > much testing on the post body sent as I wish I had. > I have also changed the ENCODING of PostWriter to "ISO-8859-1", i.e. uppercase. > I do not think it is case sensitive, but I think it is most common to use > uppercase. > See for example : > http://www.iana.org/assignments/character-sets which says "Alias: ISO-8859-1 > (preferred MIME name)" > I have also made a constant HTTP_ENCODING for both PostWriterTest and > TestHTTPSamplersAgainstHttpMirrorServer. Applied
Created attachment 19936 [details] Suggested patch to allow user to choose to use multipart/form-data for post, as well as HTTP Proxy server The suggested patch adds an option in the HTTP Request GUI, to allow the user to choose wether a HTTP POST request should use multipart/form-data or application/x-www-form-urlencoded. If a file is uploaded, the multipart/form-data is used whatever the flag is set to. This patch also changes the HTTP Proxy server, so that if the proxy sees a HTTP POST multipart/form-data request, it will insert such a request into the test plan. If this patch is applied, I think this bug can be closed as "fixed".
Patch 19936 applied to SVN in r528796
This issue has been migrated to GitHub: https://github.com/apache/jmeter/issues/1332