Bug 47622 - Allow JMeter Proxy to record HTTPS
Allow JMeter Proxy to record HTTPS
Status: RESOLVED FIXED
Product: JMeter
Classification: Unclassified
Component: HTTP
2.3.4
PC All
: P2 normal (vote)
: ---
Assigned To: JMeter issues mailing list
:
Depends on:
Blocks:
  Show dependency tree
 
Reported: 2009-08-02 07:55 UTC by Milamber
Modified: 2010-02-08 15:51 UTC (History)
1 user (show)



Attachments
Patch to add HTTPS's recording with proxy (13.15 KB, patch)
2009-08-02 07:57 UTC, Milamber
Details | Diff
This zip file contains a (fake) JMeter SSL certificat which show to browser, and start SSL connection. (2.37 KB, application/zip)
2009-08-02 07:58 UTC, Milamber
Details
Patch to add HTTPS's recording with proxy (13.85 KB, patch)
2009-08-04 16:19 UTC, Milamber
Details | Diff
Small patch to correct a twice "Connection" header on a HTTPS sampler recording by proxy (1.05 KB, patch)
2009-08-06 15:20 UTC, Milamber
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Milamber 2009-08-02 07:55:28 UTC
Element Server HTTP Proxy doesn't permit to record a navigation session on a HTTPS website.
This functionality isn't exists in JMeter.
Comment 1 Milamber 2009-08-02 07:57:16 UTC
Created attachment 24078 [details]
Patch to add HTTPS's recording with proxy
Comment 2 Milamber 2009-08-02 07:58:04 UTC
Created attachment 24079 [details]
This zip file contains a (fake) JMeter SSL certificat which show to browser, and start SSL connection. 

The file server.p12 must be put in JMETER_HOME/bin
Comment 3 Milamber 2009-08-02 07:59:50 UTC
To test :
In Proxy configuration, "Attempt HTTPS Spoofing" no need checked.
Use HTTP Request or HTTP Request HTTPClient (better)

With your browser, use same proxy (host:port) for all protocols

(If this patch is good, can be replace HTTPS spoofing features in a future release.)
Comment 4 Milamber 2009-08-02 08:03:25 UTC
I have testing this patch with Firefox 2.0 (linux), 3.0 (linux), 3.5 (windows xp) and Internet Explorer 7 and 8 (xp), no problem to record (after accepting the JMeter ssl cert)


HTTPS: Test with issues.apache.org (login, navigate)
HTTPS: Test with a Alfresco site in https (homepage, login, navigate)
HTTPS: Test with a Wordpress site with SSL authentication (using SSL Manager in JMeter during proxy record) + HTTP basic authentication, and homepage, login, dashboard, write post, display draft)
HTTPS: Gmail (login, send mail, read mail, logout)
HTTPS: Google Docs (new doc)

HTTP: Test with same Alfresco site in http (homepage, login, navigate)
HTTP: some navigation in Google News, and several news site
Comment 5 Milamber 2009-08-02 08:11:21 UTC
If you want create your ssl cert:


Generate SSL fake certificate
===================
Password (for ssl cert in zipfile): password

##### Create certificate
root@svrtest2:~#  openssl req -new -x509 -days 3652 -keyout serverkey.pem -out servercert.pem
Generating a 1024 bit RSA private key
........++++++
........................++++++
writing new private key to 'serverkey.pem'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:MA
State or Province Name (full name) [Some-State]:Rabat
Locality Name (eg, city) []:Temara
Organization Name (eg, company) [Internet Widgits Pty Ltd]:JMeter
Organizational Unit Name (eg, section) []:JMeter
Common Name (eg, YOUR name) []:Apache JMeter proxy recorder
Email Address []:webmaster@apache.org

###### export to pkcs12 format
root@svrtest2:~# openssl pkcs12 -export -in servercert.pem -inkey serverkey.pem -name "Apache JMeter proxy recorder" -clcerts -out server.p12
Enter pass phrase for serverkey.pem:
Enter Export Password:
Verifying - Enter Export Password:
Comment 6 Milamber 2009-08-04 16:19:37 UTC
Created attachment 24103 [details]
Patch to add HTTPS's recording with proxy

I upgraded the patch to include in file 'build.xml' the server.p12 in the ZIP/TGZ package construction,
and modified a log message when a ssl request is canceled by browser (ca_unknown / anti-phishing method).
Comment 7 Sebb 2009-08-05 17:04:55 UTC
Thanks very much for this work - it's fantastic to have HTTPS recording at last.

I made some minor changes to the patch:
- Generated a new certificate using the Java keytool
- added scripts to recreate the keystore
- Renamed store as proxserver.jks
- added some more properties to configure the keystore
- tidied up the error handling a bit
- added some documentation

Applied as 

URL: http://svn.apache.org/viewvc?rev=801473&view=rev
Log:
Bug 47622 - enable recording of HTTPS sessions
Many thanks to Milamber.

It will be in nightly builds after r801473.
Comment 8 Milamber 2009-08-05 23:42:54 UTC
Thanks for your improvements and integration's patch.
I'm very happy too for have this functionality in my favorite tool!
Comment 9 Milamber 2009-08-06 15:20:33 UTC
Created attachment 24114 [details]
Small patch to correct a twice "Connection" header on a HTTPS sampler recording by proxy

When proxy record a https request, it's put a "Connection: keep-alive" header in Headers Manager, but this header is already manage (or not) by HTTP sampler's keep-alive option.
In a HTTP request, the "Connection" header is "Proxy-Connection: keep-alive
", which is already excluded

(and unused import removed)
Comment 10 Sebb 2009-08-11 06:42:24 UTC
Thanks, added to SVN:

URL: http://svn.apache.org/viewvc?rev=803117&view=rev
Log:
Bug 47622 - don't add Connection header from browser
Comment 11 Arijit 2010-01-06 02:47:54 UTC
Hi

I was trying to use this patch to record an https login (to our product interface).  In the browser, I had to accept the certificate, and everything worked ok - but when the recorded script is rerun, all I get (in the server response) is the login screen - and this is happening for gmail too.

Not sure whether I should have posted it here - but this is the only relevant link I found. Sorry if this isn't the right place.

Regards
Arijit
Comment 12 Milamber 2010-01-06 14:24:26 UTC
Your problem seems be a user sessions status mecanism in your application. Without link with HTTPS proxy. (thus the best place to this problem is the JMeter user list http://jakarta.apache.org/site/mail2.html#JMeter)
You must have a "HTTP Cookie Manager" in your JMeter test plan for the ID session mechanism, which is used to look up authenticated session. Or some Regular expression post-processor to extract "view state" mecanism from your application.
Please, check all HTTP parameters / cookies for find sessions status's mecanisms.

Milamber
Comment 13 Arijit 2010-01-06 22:43:43 UTC
(In reply to comment #12)
> Your problem seems be a user sessions status mecanism in your application.
> Without link with HTTPS proxy. (thus the best place to this problem is the
> JMeter user list http://jakarta.apache.org/site/mail2.html#JMeter)
> You must have a "HTTP Cookie Manager" in your JMeter test plan for the ID
> session mechanism, which is used to look up authenticated session. Or some
> Regular expression post-processor to extract "view state" mecanism from your
> application.
> Please, check all HTTP parameters / cookies for find sessions status's
> mecanisms.
> 
> Milamber

Thank you. I'll check the user list. One more question - is this patch now part of the stable build? Or should I use the latest nightly builds? I mean I couldn't find the releases mentioned in this thread (except the source files in SVN) - I was wondering whether the latest nightly builds will contain this patch or not...
Comment 14 Milamber 2010-01-06 23:39:17 UTC
This patch will come with the next JMeter release (2.4) - no release date planned. 
Since this patch is commited in SVN, it is in all nightly builds
Tips: You can use lastest nightly builds to record scenario in HTTPS, and use a version 2.3.4 for run a load tests.
Comment 15 Victor Klepikovskiy 2010-02-08 14:47:57 UTC
I found a small bug - if I run JMeter and the current directory is not jmeter/bin, HTTPS recording feature does not work, seems JMeter can not read certificate file in this case.
Comment 16 Sebb 2010-02-08 15:51:09 UTC
Thanks for the report. The code has been changed to default to the JMeter bin directory instead of the current working directory:

URL: http://svn.apache.org/viewvc?rev=907847&view=rev
Log:
Bug 47622 - dummy JMeter certificate resides in the bin directory
Document the properties

Modified:
   jakarta/jmeter/trunk/src/protocol/http/org/apache/jmeter/protocol/http/proxy/Proxy.java
   jakarta/jmeter/trunk/xdocs/usermanual/component_reference.xml