In the current implementaion, scp task included in Apache Ant has been using scp1 protocol to transfer files, but some proprietary sshd[1] has not supported it anymore. To resolve this problem, scp task should use sftp protocol[2], which must be supported by almost of all ssh2 sshd. [1] http://www.snailbook.com/faq/scp-ossh-to-ssh2.auto.html [2] http://www.unix.org.ua/orelly/networking_2ndEd/ssh/ch03_08.htm
Created attachment 18147 [details] a patch to use sftp protocol in scp task. Attached patch will allow scp task to use sftp protocol instead of scp1 protocol. If you add the attribute 'sftp="true"', sftp protocol will be used and if 'sftp="false"', scp1 protocol will be used. By the default, the attribute 'sftp' is "false" for backword compatibity. To try this patch, jsch-0.1.27[1] is required. [1] http://sourceforge.net/project/showfiles.php?group_id=64920
Wouldn't it be better to define a new task called "sftp" rather than changing the "scp" task to no longer use scp?
+1 on Ben's comment
-1 on Ben's comment. Please read the links provided by Atsuhiko, particularly [2] http://www.unix.org.ua/orelly/networking_2ndEd/ssh/ch03_08.htm. SFTP is the protocol used by the program scp2. It is entirely appropriate to add this feature to the scp task.
After applying the patch and running a basic test against an F-Secure sshd I get the following error: BUILD FAILED H:\build.xml:6: com.jcraft.jsch.JSchException: invalid server's version string I am using the jsch 0.1.27
(In reply to comment #5) > After applying the patch and running a basic test against an F-Secure sshd I > get the following error: > BUILD FAILED > H:\build.xml:6: com.jcraft.jsch.JSchException: invalid server's version string > I am using the jsch 0.1.27 Does that sshd support ssh2 protocol?
(In reply to comment #6) > (In reply to comment #5) > > After applying the patch and running a basic test against an F-Secure sshd I > > get the following error: > > BUILD FAILED > > H:\build.xml:6: com.jcraft.jsch.JSchException: invalid server's version > string > > I am using the jsch 0.1.27 > Does that sshd support ssh2 protocol? The version of F-Secure sshd we are using supports only ssh2 protocol.
(In reply to comment #7) > The version of F-Secure sshd we are using supports only ssh2 protocol. It is interesting. May I ask you to show me the fist line of outputs from $ telnet address_of_sshd 22
(In reply to comment #8) > (In reply to comment #7) > > The version of F-Secure sshd we are using supports only ssh2 protocol. > It is interesting. > May I ask you to show me the fist line of outputs from > $ telnet address_of_sshd 22 SSH-2.0-3.2.9 F-SECURE SSH 3.2.3
(In reply to comment #9) > (In reply to comment #8) > > (In reply to comment #7) > > > The version of F-Secure sshd we are using supports only ssh2 protocol. > > It is interesting. > > May I ask you to show me the fist line of outputs from > > $ telnet address_of_sshd 22 > SSH-2.0-3.2.9 F-SECURE SSH 3.2.3 It appears that the ascii codes for the server version string returned by F- Secure is lower case. The following patch to Session.java will address the issue. --- Session.java.orig 2006-04-18 22:37:02.000000000 -0700 +++ Session.java 2006-04-24 15:42:58.594132100 -0700 @@ -224,11 +224,13 @@ } } - if(i==buf.buffer.length || + if((i==buf.buffer.length || (i>4 && (buf.buffer[0]!='S'||buf.buffer[1]!='S'|| buf.buffer[2]!='H'||buf.buffer[3]!='-'))|| i<7 || // SSH-1.99 or SSH-2.0 - (buf.buffer[4]=='1' && buf.buffer[6]!='9') // SSH-1.5 + (buf.buffer[4]=='1' && buf.buffer[6]!='9')) ^ // SSH-1.5 + (buf.buffer[0]!='s'||buf.buffer[1]!='s'|| + buf.buffer[2]!='h'||buf.buffer[3]!='-') // "ssh-" F-Secure returns lowercase ){ throw new JSchException("invalid server's version string"); } The task is still not working however. Now I get the following error... com.jcraft.jsch.JSchException: Session.connect: java.io.IOException: invalid data This exception is thrown during the Session.read(buf) method.
(In reply to comment #10) > (In reply to comment #9) > > SSH-2.0-3.2.9 F-SECURE SSH 3.2.3 > > It appears that the ascii codes for the server version string returned by F- > Secure is lower case. The following patch to Session.java will address the > issue. It is curious. You had written as in comment #9 SSH-2.0-3.2.9 F-SECURE SSH 3.2.3 , which starts with upper case string "SSH-" and the RFC4253[1] says at Section 4.2 as follows, 4.2. Protocol Version Exchange When the connection has been established, both sides MUST send an identification string. This identification string MUST be SSH-protoversion-softwareversion SP comments CR LF Since the protocol being defined in this set of documents is version 2.0, the 'protoversion' MUST be "2.0". The 'comments' string is OPTIONAL. If the 'comments' string is included, a 'space' character (denoted above as SP, ASCII 32) MUST separate the 'softwareversion' and 'comments' strings. I don't have the reason why it starts with 'ssh-'. > The task is still not working however. Now I get the following error... > com.jcraft.jsch.JSchException: Session.connect: java.io.IOException: invalid data > This exception is thrown during the Session.read(buf) method. If it really sents 'ssh-', it may not use formal ssh2 protocol, IMHO. Anyway, that problem is not related to this topic. [1] http://www.ietf.org/rfc/rfc4253.txt
(In reply to comment #11) > (In reply to comment #10) > > (In reply to comment #9) > > > SSH-2.0-3.2.9 F-SECURE SSH 3.2.3 > > > > It appears that the ascii codes for the server version string returned by F- > > Secure is lower case. The following patch to Session.java will address the > > issue. > It is curious. You had written as in comment #9 > SSH-2.0-3.2.9 F-SECURE SSH 3.2.3 > , which starts with upper case string "SSH-" and the RFC4253[1] says at Section 4.2 as follows, > 4.2. Protocol Version Exchange > > When the connection has been established, both sides MUST send an > identification string. This identification string MUST be > SSH-protoversion-softwareversion SP comments CR LF > Since the protocol being defined in this set of documents is version > 2.0, the 'protoversion' MUST be "2.0". The 'comments' string is > OPTIONAL. If the 'comments' string is included, a 'space' character > (denoted above as SP, ASCII 32) MUST separate the 'softwareversion' > and 'comments' strings. > I don't have the reason why it starts with 'ssh-'. > > The task is still not working however. Now I get the following error... > > com.jcraft.jsch.JSchException: Session.connect: java.io.IOException: invalid data > > This exception is thrown during the Session.read(buf) method. > If it really sents 'ssh-', it may not use formal ssh2 protocol, IMHO. > Anyway, that problem is not related to this topic. > [1] http://www.ietf.org/rfc/rfc4253.txt I agree that this is a seperate issue. I would like to work with you to resolve it, if possible. I am interested in the enhancements to the scp task and would like to see them committed to the Ant project. Should we take this discussion to the ant-dev list?
Created attachment 18186 [details] an implementation of sftp task The attached patch is a quick hack to implement sftp task, which is based on ftp task and using jsch for sftp functionality. It accepts attributes for ftp and scp tasks except for 'binary', 'passive', 'separator' and 'umask'. It seems following tasks are working on my testing, however, frankly to say, I'm not familiar with ftp task and have never used and run it, so I may mis-understand its operational semantics. Comments, suggestions and feedbacks are welcome. PS 1. that patch is for ant_20060426105634.tar.gz PS 2. jsch-0.1.27 is required. <sftp action="send" server="${server}" userid="${userid}" password="${password}" remotedir="incoming" depends="yes"> <fileset dir="htdocs/manual"/> </sftp> <sftp action="get" server="${server}" userid="${userid}" password="${password}"> <fileset dir="htdocs/manual" > <include name="**/*.html"/> </fileset> </sftp> <sftp action="del" server="${server}" userid="${userid}" password="${password}" > <fileset> <include name="**/*.tmp"/> </fileset> </sftp> <sftp action="list" server="${server}" userid="${userid}" password="${password}" listing="data/sftp.listing" > <fileset> <include name="**"/> </fileset> </sftp> <sftp action="mkdir" server="${server}" userid="${userid}" password="${password}" remotedir="test" /> <sftp action="chmod" server="${server}" userid="${userid}" password="${password}" chmod="0700" > <fileset> <include name="*.log"/> </fileset> </sftp>
Why the change in direction? I like the idea of adding the sftp attribute to the scp task, rather than creating a new sftp task. Perhaps both are a good idea.
(In reply to comment #14) > Why the change in direction? I like the idea of adding the sftp attribute to > the scp task, rather than creating a new sftp task. Perhaps both are a good > idea. Of course, that patch includes both.
After working out an issue with Jsch, I have successfully tested the enhancements to the scp task. I think the sftp task enhancement should be in a seperate bug report, because I believe it is a different subject. Furthermore, I feel that this enhancement to the scp task is a higher priority than the need for a new sftp task.
(In reply to comment #5) > After applying the patch and running a basic test against an F-Secure sshd I > get the following error: > BUILD FAILED > H:\build.xml:6: com.jcraft.jsch.JSchException: invalid server's version string > I am using the jsch 0.1.27 Thanks to Robert, I found the problem in the jsch code and has fixed it, at last. If somebody has simillar problem, http://www.jcraft.com/jsch/jsch-0.1.28- rc4.jar will resolve that problem.
(In reply to comment #16) > I think the sftp task enhancement should be in a seperate bug report, because I > believe it is a different subject. Furthermore, I feel that this enhancement to > the scp task is a higher priority than the need for a new sftp task. I agree with him. There should be a separate bug report for sftp task, but that patch for sftp task is based on the source code for this modified scp task. Dear ant developers, are there not any possibilties that this modification for scp task will be adopted? If so, I'll delete it from the source code and submit a new bug entry for sftp task and post a patch, which does not include the modifications for scp. I don't want to keep several source code trees.
I've committed the part of the patch that adds sftp support to scp, the sftp task will remain separate since it needs a bit more of review than I can do right now. You'll note that there are a few differences between your patch and what I committed, most of them are whitespace only. We also need an update of scp's manual page. IIUC we should recommend to set the sftp attribute to true if the ssh server supports ssh2, correct?
Created attachment 18251 [details] Patch to scp.html manual page
(In reply to comment #19) > We also need an update of scp's manual page. IIUC we should recommend to set > the sftp attribute to true if the ssh server supports ssh2, correct? IMHO, if users have satisfied with original scp task, there are not any merit to use sftp attribute, becase scp1 protocol is more efficent than sftp protocol. So, I hesitate to recommend to use it explicitly. That attribute is only for some users, whose sshd does not support scp1 anymore. Almost users will use OpenSSH's sshd and it will continue to support scp1 for a while.
(In reply to comment #20) > Patch to scp.html manual page I've applied a slightly modified version (recommending the option if the server doesn't support SSH1 based on Atsuhiko Yamanaka's comment) but svn is down right now so I can't commit it.
(In reply to comment #22) > I've applied a slightly modified version (recommending the option if the server > doesn't support SSH1 based on Atsuhiko Yamanaka's comment) but svn is down right > now so I can't commit it. I have meant scp1 protocol, not ssh1 protocol. Sorry for lack of words. scp1 protocol is just a file transfer protocol and it has been used on ssh1 and ssh2 protocol. For example, openssh'd sshd can disable ssh1 protocol support by configuring /etc/ssh/sshd_conifg, but scp1 procol can be available if there exists '/usr/bin/scp' command.
Stefan, have you upped the manual change? Can this one be closed?
See bug 39532 for a new sftp task patch, closing this one.
Created attachment 18921 [details] bugfix to download files/directory, which are pointed by symbolic link. I have found a bug in downloading files and directories, which are pointed by symbolic links, with scp task using sftp of ant 1.7.0beta2. I will attache a patch to fix this problem.
*** Bug 39532 has been marked as a duplicate of this bug. ***