[NIFI-9509] Upgrade to SSHJ 0.32 introduces a SFTP renaming issue with AWS SFTP Transfer Family

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.15.0, 1.15.1
• Fix Version/s: 1.16.0, 1.15.2
• Component/s: Extensions
• Environment: Apache NIFI Docker image tag 1.15.1.
• Assignee: David Handermann
• Reporter: Steve
• Created: 21/Dec/21 11:24
• Updated: 20/Apr/22 13:14
• Resolved: 21/Dec/21 19:12

Description

We have a NIFI process that uses the SFTP processor to put files to an AWS Transfer family SFTP  server. The "dot file rename" option is enabled on this processor.

This has been working for some time on NIFI 1.14.0 and following the upgrade to 1.15.1 it now continually fails with a read timeout.  If the "dot file rename" option is disabled on this processor then the SFTP put works ok.

This appears to be an issue with the underlying SSHJ 0.32 that was upgraded in NIFI-9303. Release 0.32 includes support for SFTP rename flags which as a SFTP V5+ feature. Unfortunately SSHJ supports a max version of 3 so there is a bit of unpicking to be had here. The impact is that all rename packets will contain a rogue uint32 at the end, which may or may not cause issues depending on how the server handles badly formed or unexpected content in V3 packets.

There are two issues on SSHJ GitHub issues relating to this.

https://github.com/hierynomus/sshj/issues/751

https://github.com/hierynomus/sshj/issues/750

 

Example failure log attached.

Issue Links

• relates to: NIFI-9939 Upgrade SSHJ to 0.33.0
• links to: GitHub Pull Request #5618
• links to: SSHJ PR 754