[CAMEL-16692] SFTP sometimes doesn't receive all files

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.7.3
• Fix Version/s: 3.11.1, 3.12.0
• Component/s: camel-ftp

Environment

I'm reproducing this on my own computer and a SFTP server running on a Docker container locally.

• Estimated Complexity: Unknown
• Assignee: Claus Ibsen
• Reporter: Jonas Audenaert
• Created: 07/Jun/21 07:38
• Updated: 29/Jun/21 05:25
• Resolved: 24/Jun/21 13:13

Description

I have following Camel route, created by a RoutingBuilder:

new org.apache.camel.builder.RouteBuilder() {
  void configure() {
    from("direct:myProgram")
      .routeId("${params.profile}")
      .to("sftp://${params.hostname}:${params.port}/${params.directory}?username=RAW(${params.username})&password=RAW(${params.password})&fileName=RAW(\${header.fileName})&fileExist=Fail&disconnect=true&stepwise=false&useUserKnownHostsFile=false&serverAliveInterval=${params.serverAliveInterval}")
  }
}

This route is the only one which I place on my CamelContext and than I start the CamelContext.

I'm doing a test on my local machine to send 50 files through this route to a remote SFTP server. Most of the time, all files arrive, but sometimes 1 file seems to be missing.

My program doesn't notice the failure of sending. But during debugging I found something interesting that happened.

That one specific file which is missing throws in the camel library an SFTPException at the method existsFile (SftpOperations.java:1046).

The exception contains only reason 4: message but the exception contains another exception. The wrapped exception contain following exception message: java.io.IOException: Pipe Closed.

Camel wraps the exception in a GenericFileOperationFailedException and throws it to the method which called SftpOperations, GenericFileProducer (GenericFileProducer.java:135).
That exception arrives in the tr/catch block of it and goes to the generic catch block.

In that catch-block the exception is handled by the handleWrite() method (RemoteFileProducer.java:83) and there it gets interesting.
In handleWrite() check is performed if the Camel Base Service is in stopping() or stopped() state. If that's the case, the exception is only logged but not thrown. But isStopped() returns true, so my exception is thrown, but my application doesn't receive it, so it thinks that the file is transferred succesfully to the SFTP server, but it never arrives.