Uploaded image for project: 'MINA SSHD'
  1. MINA SSHD
  2. SSHD-1078

SCP/SFTP failed for big file upload

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.5.1
    • 2.6.0
    • None

    Description

      Trying to merge product from jsch to mina sshd that keep the same SSH architecture (lets call SessionPool) which for each remote node, there is only one session object and limited channels (<=8) for executing commands and file operations. The SessionPool  manages the session/channels for nodes  and make sure only 8 threads(ssh channels) per node that can be run concurrently. This architecture works for Jsch for many years, but when using mina sshd, this architecture wont work when  with upload big file, we always see following exception:

      java.io.EOFException: Channel is being closed
      at org.apache.sshd.common.channel.AbstractChannel$2.<init>(AbstractChannel.java:794)
      at org.apache.sshd.common.channel.AbstractChannel.writePacket(AbstractChannel.java:792)
      at org.apache.sshd.common.channel.ChannelAsyncOutputStream.doWriteIfPossible(ChannelAsyncOutputStream.java:168)
      at org.apache.sshd.common.channel.ChannelAsyncOutputStream.writePacket(ChannelAsyncOutputStream.java:70)
      at org.apache.sshd.client.subsystem.sftp.impl.DefaultSftpClient.send(DefaultSftpClient.java:292)
      at org.apache.sshd.client.subsystem.sftp.impl.SftpOutputStreamAsync.flush(SftpOutputStreamAsync.java:210)
      at org.apache.sshd.client.subsystem.sftp.impl.SftpOutputStreamAsync.write(SftpOutputStreamAsync.java:141)
      

      From SSH server log, we see follow error:

      Aug 19 15:56:38 xxxx sftp-server[202217]: error: Unknown message 0
      Aug 19 15:56:38 xxxx sftp-server[202217]: error: msg_len 0 < consumed 1
      

      other client side exceptions by mina sshd as:
      org.apache.sshd.common.channel.WindowClosedException: Already closed: Window[client/remote](ChannelExec[id=79, recipient=1]-ClientSessionImp

       


      After many tries, finally I found that for big file upload/download, I have to give its own SshClient/ClientSession objects and then there will  no such channel closing exception during file upload.  

      That means we cannot share the same SshClient and ClientSession object per remote node to execute commands and file operations, and there may has bug in ClientSession when handle channels in this situation.

      the reason that we like to share the same SshClient/ClientSession for the same remote node is to manage the session/channels and close them when no usage, also to reduce the cost to allocate those objects. 

       

       

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              minlaw min yun law
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: