Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7049

SFTP processors shouldn't silently try to access known hosts file of the user



    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.12.0, 1.11.3
    • Component/s: Extensions
    • Labels:


      In case NiFi test are executed on a machine without knows_hosts file, it's going to fail:

      [INFO] Running org.apache.nifi.processors.standard.TestGetSFTP
      [ERROR] Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.374 s <<< FAILURE! - in org.apache.nifi.processors.standard.TestGetSFTP
      [ERROR] testGetSFTPFileBasicRead(org.apache.nifi.processors.standard.TestGetSFTP)  Time elapsed: 0.132 s  <<< FAILURE!
      java.lang.AssertionError: expected:<4> but was:<0>
      	at org.apache.nifi.processors.standard.TestGetSFTP.testGetSFTPFileBasicRead(TestGetSFTP.java:88)
      [ERROR] testGetSFTPIgnoreDottedFiles(org.apache.nifi.processors.standard.TestGetSFTP)  Time elapsed: 0.013 s  <<< FAILURE!
      java.lang.AssertionError: expected:<2> but was:<0>
      	at org.apache.nifi.processors.standard.TestGetSFTP.testGetSFTPIgnoreDottedFiles(TestGetSFTP.java:110)

      Just pasting my private message that summarised this error previously:

      So the problem is that host key file is not a mandatory, but in case it’s not provided, we call load on the 3rd party lib without arguments:
      Which tries to load keys from the default location, but this is far from what we state in our documentation:

      Host Key File            If supplied, the given file will be used as the Host Key; otherwise, no use host key file will be used 

      So there are multiple issues here:
      -Even though the ssh connection fails, somewhere the IO exception is swallowed. Didn’t reproduce to check the logs, but I would expect exceptions to be thrown in the testcase and these being talkative about the error. My gut feeling says that we do the same in case the user specifies a host key file, but it’s somehow not accessible.
      -Strict host check on/off might not be enough to cover all the scenarios as there are three: host 1# known and key matches, 2# host not known and we either trust or not, 3# host known, but there is a mismatch (probably man in the middle). I think this property should be improved at least in documentation point of view as currently only the code tells what do we do in 2#. Which depends on whether the file exists or not, so most probably something unintended.
      -Either the documentation or the behaviour should be fixed to make them aligned
      -The testcase should either use a predefined key or have host key checking completely off. According to what we see above, not sure about the latter being nicely supported.


          Issue Links



              • Assignee:
                Dayakar Dayakar Morem
                aboda Arpad Boda
              • Votes:
                0 Vote for this issue
                3 Start watching this issue


                • Created:

                  Time Tracking

                  Original Estimate - Not Specified
                  Not Specified
                  Remaining Estimate - 0h
                  Time Spent - 1h