NTLMv2 support

• Key: HTTPCLIENT-579
• Type: Improvement
• Status: Closed
• Resolution: Won't Fix
• Priority: Major
• Assignee: Unassigned
• Reporter: Konstantin Kasatkin
• Component/s: HttpAuth
• Affects Version/s: 3.0 Final
• Fix Version/s: 3.1 Alpha 1

Description

Hello guys,
recently I've encouner the problem how to interact with HTTP resourses protected tough security policy allowed to use only NTLMv2 authorization scheme from a java application.

Earlier I used httpclient to accomplish NTLMv1 authorization, but when I was requested to move to NTLMv2 I was surprised, that httpclient does not support this.

I've dug all the Internet and found some articles and examples how to implement it, and finally having compiled all this enormous heap of information, I have a couple modified classes from httpclient project, that I'd like to share with you and other users of httpclient.

Comments

[Konstantin Kasatkin]

I've used commonly 3 sources of knowledge.

http://davenport.sourceforge.net/ntlm.html - The NTLM Authentication Protocol by Eric Glass, 2003
http://sourceforge.net/projects/jtds - jTDS - SQL Server and Sybase JDBC driver
http://ubiqx.org/cifs/index.html - The book "Implementing CIFS: The Common Internet FileSystem" by Christopher R. Hertel, 2003

Improvement codes are in attachment.

[Ortwin Glück]

Licensing:

The bouncycastle code allows relicensing and redistribution. So, no problem.
The DESEngine's origin is unclear. Please tell where this code comes from and under which license it is released.
For our own classes please provide them in unidiff (diff -u) format against SVN trunk.

You say that your work is based on JTDS and Davenport. Both these projects are LGPL which could be a problem as your work may be a "derived work" and can thus not be licensed under the ASF license.

[Oleg Kalnichevski]

Well, some bad news. jTDS is licensed under LGPL. There is no way we can take any LGPL licensed code in any form or fashion into a official Apache distribution. Apache license is incompatible with LGPL. There is nothing we can do about it

There is only one option left, though. You should start a new project on Sourceforge or Javaforge and release the code under LGPL. In our turn we can provide a link on our site to this project and include documentation how to plug the NTLMv2 auth module in and configure HttpClient to use it instead of the standard NTLMv1 one

Sorry

Oleg

[Oleg Kalnichevski]

Konstantin,
Alternatively consider implementing an NTLM authentication module based on a third party library such as JCIFS [1]. It will be much easier for us to accept an original contribution that directly relies on a third party library rather than re-uses various bits and pieces of a third party library without an original author(s) consent.

Oleg

[1] http://jcifs.samba.org/

[Konstantin Kasatkin]

   [[ Old comment, sent by email on Fri, 19 May 2006 22:55:53 +0400 ]]

Oleg,
as for JCIFS library, I can't understand what prevents you to use its API.

[Roland Weber]

Hello Konstantin,

we have an NTLM FAQ now. Your question is #4.
http://wiki.apache.org/jakarta-httpclient/FrequentlyAskedNTLMQuestions

cheers,
  Roland

[Ortwin Glück]

There is more recent information here (mind the apostrophe):

http://wiki.apache.org/jakarta/Using_LGPL'd_code

see especially bottom of page.

We are allowed to use JCIFS if we follow the rules. It's just a matter of someone doing the job.

Odi

[Zhao Cheng (zcheng)]

Hello guys,
 
Hope I am sending my question to the right alias. Recently I am looking
for a way to support NTLMv2 using HttpClient library, and came across
the following link.
http://issues.apache.org/jira/browse/HTTPCLIENT-579
 
Looks like a guy named Konstantin Kasatkin implemented support for
NTLMv2, I downloaded NTLMv2.rar file as he attached in the email
discussion in the above link. After download, I opened the rar file, and
compiled it, then added class files from the compilation to the original
commons-httpclient-3.0.jar file, but somehow after this change, my code
can not even talk to NTLMv1 which was working earlier in my code.
 
I am wondering what might be wrong with my approach? any help or hints
are greatly appreciated.
 
thanks
 
Zhao
 
 
 
 

[Oleg Kalnichevski]

Zhao,

We were not able to accept the code submitted by Konstantin due to the licensing issues, as some bits of it were based on LGPL licensed code. Therefore none of us actually tested the contributed code.

Oleg

[Zhao Cheng (zcheng)]

Hello guys,
 
Hope I am sending my question to the right alias. Recently I am looking
for a way to support NTLMv2 using HttpClient library, and came across
the following link.
http://issues.apache.org/jira/browse/HTTPCLIENT-579
 
Looks like a guy named Konstantin Kasatkin implemented support for
NTLMv2, I downloaded NTLMv2.rar file as he attached in the email
discussion in the above link. After download, I opened the rar file, and
compiled it, then added class files from the compilation to the original
commons-httpclient-3.0.jar file, but somehow after this change, my code
can not even talk to NTLMv1 which was working earlier in my code.
 
I am wondering what might be wrong with my approach? any help or hints
are greatly appreciated.
 
thanks
 
Zhao
 
 
 
 

[Zhao Cheng (zcheng)]

Is there a way that I can contact Konstantin? I am really interested in
figuring out how he made it work.
 
thanks
 
Zhao

[Ortwin Glück]

Click on his name in the title of his JIRA comments. This will bring up his user profile page wich includes an email address.

[Konstantin Kasatkin]

Hi Zhao, I'm sure you can contact:)

I think you've made something wrong in compilation process because the code I'd attached successfully works in production nowadays.

[V. K.]

Hello Konstantin,

I am from Greece and I 've tried to use your code. I downloaded the source code of HttpClient-3.0, I replaced the files NTLM.java, NTLMauth.java and NTLMScheme.java with your own ones and I also added the files under the org\apache\commons\httpclient\crypto package that you provided. After that, I compiled all the files using jdk1.6.0_01 java platform and I used the following code to call a web page (which I found as an example into the HttpClient-3.0 source code):
__________________________________________________________________________________

ProxyClient proxyclient = new ProxyClient();
        // set the host the proxy should create a connection to
        //
        // Note: By default port 80 will be used. Some proxies only allow conections
        // to ports 443 and 8443. This is because the HTTP CONNECT method was intented
        // to be used for tunneling HTTPS.
        proxyclient.getHostConfiguration().setHost("www.verisign.com", 443);
        // set the proxy host and port
        proxyclient.getHostConfiguration().setProxy("myproxy", 8080);
        // set the proxy credentials, only necessary for authenticating proxies
        proxyclient.getState().setProxyCredentials(
            new AuthScope("myproxy", 8080, null),
            new UsernamePasswordCredentials("myWindowsUsername", "myWindowsPwd"));
        
        // create the socket
        ProxyClient.ConnectResponse response = proxyclient.connect();
        
        if (response.getSocket() != null) {
            Socket socket = response.getSocket();
            try {
                // go ahead and do an HTTP GET using the socket
                Writer out = new OutputStreamWriter(
                    socket.getOutputStream(), "ISO-8859-1");
                out.write("GET http://www.verisign.com/ HTTP/1.1\r\n");
                out.write("Host: www.verisign.com\r\n");
                out.write("Agent: whatever\r\n");
                out.write("\r\n");
                out.flush();
                BufferedReader in = new BufferedReader(
                    new InputStreamReader(socket.getInputStream(), "ISO-8859-1"));
                String line = null;
                while ((line = in.readLine()) != null) {
                    System.out.println(line);
                }
            } finally {
                // be sure to close the socket when we're done
                socket.close();
            }
        } else {
            // the proxy connect was not successful, check connect method for reasons why
            System.out.println("Connect failed: " + response.getConnectMethod().getStatusLine());
            System.out.println(response.getConnectMethod().getResponseBodyAsString());
        }
    }

__________________________________________________________________________________


The problem is that I still receive the error 'Connect failed: HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy service is denied.' which is the same one that I had when I used the initial HttpClient-3.0.
 Could you please tell me which version of HttpClient did you use? There are plenty of them starting from 2.0-beta1 till 4.0-alpha1. Moreover, in case you noticed anything strange in the above code, could you please give me a sample of java code that authenticates a user to a proxy server?

Thanks in advance for your reply,
Voula

[Roland Weber]

This is an issue tracking system, not a bulletin board.
Please use the user mailing list for discussions.
http://jakarta.apache.org/commons/httpclient/mail-lists.html
http://mail-archives.apache.org/mod_mbox/jakarta-httpclient-user/

cheers,
  Roland

[Konstantin Kasatkin]

Hi guys!
Sorry for long silence.

I use http client v3.0 and it works fine.
It's not a problem for me to provide a totally functioning example,
but I can't attach example in this topic because it's closed.

Just send me your emails in a private message and I'll share the example.

[Oleg Kalnichevski]

Konstantin

Please consider hosting your code at the SourceForge or any other hosting site. As I said before we are not able to accept LGPL derived work into an ASF owned codeline, but we can advise the users to use the auth scheme you have developed as a recommended add-on and could even potentially deprecate our own NTLM atuh scheme in favor of yours.

Oleg

[Andrew Franklin]

Konstantin,

I've added a patch to the code you provided above to handle the case in Windows Server 2003 where "Require NTLMv2 session security" is set as an option inside "Network security: Minimum session security for NTLM SSP based (including secure RPC) servers".

The changes I made were a result of reading of: http://davenport.sourceforge.net/ntlm.html + experiments through WireShark so while I'd happily licence my change under the Apache Licence, I don't believe it's allowed as the original patch was under the LGPL.

Maybe it is possible as per Oleg's request that we can create an external project (on sourceforge or wherever) with a LGPL licence?

Andrew

[kay lin]

I'm running the same issue like Zhao had. What was the resolution? Thanks.
Kay.

[Oleg Kalnichevski]

What's unclear about resolution 'Won't Fix '?

However, HttpClient 4.0 is capable of supporting NTLMv2 through JCIFS library. For details see

http://hc.apache.org/httpcomponents-client/ntlm.html

Oleg

[kay lin]

Thanks for the reply. If I must stick with httpclient 3.0.1, what are the suggestions to work around the problem besides purchasing a 3rd party software?
Thanks.
Kay.

[Ortwin Glück]

Kay, you can always patch your local version of HttpClient 3 with the patches above. It's not illegal. It's just that you create a package that is probably no longer ASF-2 licensible. That's the reason why we cannot provide this out of the box. What kind of effect this will have on your final product's license terms you will need to check with a software licensing expert :-)

[kay lin]

Thanks Ortwin. Any suggestions on making httpclient 3.x to work with axis2 to support ntlm v2?
Thanks,
Kay.

[Konstantin Kasatkin]

Finally I've decided to attach complete example of integration Axis, HttpClient and NTLM2.
Use it for your own.

[Urvi]

kasatkin,

Please provide ../lib/commons-httpclient-3.0-ntlm-2.0.jar file. In above NTLMV2.rar file you have old version of httpclient file.

I would really appriciate.

Thanks!

[Reflex Demon]

Kasatkin,

Me too waiting for this release.

If you can post us your commons-httpclient-3.0-ntlm-2.0.jar it would help us.


Thank in advance!

Regards,
Reflex

[Konstantin Kasatkin]

Guys, I'd recommend you before sending your requests to figure out how src/build.xml really works.
It unpacks original commons-httpclient-3.0.jar, substitutes necessary classes and packs compilled ones into commons-httpclient-3.0-ntlm-2.0.jar.

Have you tried just to run build.xml, what's the problem?

[Reflex Demon]

Kasatkin,


I had a look at the build.xml file.
It looks like this.

<project name="casau" default="main">
  <target name="main" >
    <javac srcdir="." destdir="." deprecation="yes" debug="yes"
           classpath="../lib/commons-logging-1.0.4.jar;../lib/commons-codec-1.3.jar;../lib/commons-httpclient-3.0-ntlm-2.0.jar">
      <include name="**/*.java"/>
    </javac>
  </target>
</project>


Anyhow thanks for sharing the logic to build the commons-httpclient-3.0-ntlm-2.0.jar


Regards,
Reflex

[Reflex Demon]

Kasatkin.


I found the corret build.xml in the src folder.

Thanks a lot man.


Regards,
Reflex

[Konstantin Kasatkin]

It's just artefact, you can remove it if it disturbs you:)

Just run build.xml and enjoy.

[Reflex Demon]

Kasatkin,

One quick question, will this work for other NTLM authentication?

Regards,
Reflex.

[Konstantin Kasatkin]

What do you mean under "other NTLM authentication" term?

BTW, my first name is Konstantin:)

[Reflex Demon]

I ment the old authentication method NTLMv1

[Konstantin Kasatkin]

Actually no, because I've had little time to implement the algorithm and there have been no time to add features for seamless switching to different versions NTLM. The original library httpclient has already included NTLMv1 implementation so you can use its implementation for interaction via NTLMv1.
All what you need is to implement a method that could get NTLM version from HTTP request and basing on the result use corresponding authentification algorithm.

[Urvi]

Konstantin,

Thanks! We got it working.. This Jar has saved our life :-)

[Reflex Demon]

Kasatkin ,

Is there anything specific to Unix on the implementation.

My question is, if I put the new compiled jar file on unix server and try to login into NTLMv2 server via the HTTP client will that login or fail?


Regards,
Reflex

[Konstantin Kasatkin]

Do you see any native libraries to be required for httpclient?:)
The main idea of this library is to work anywhere.

Certainly you can use it under any java compatible environment.

[mea]

Hi Konstantin,

I'm using "commons-httpclient-3.0-ntlm-2.0.jar" (and AXIS) to authenticate ws on windows server with NTLMv1.
It works! But you said it works only NTLMv2. How is possible?

Tomorrow I will try it with NTLMv2.

Thanks a lot.
M