When multiple putHDFS processors exist, the first to run loads some config that is then used by other putHDFS processors that are started.
I have two dataflows setup. One pushes data to a kerberized HDFS cluster while the other pushes data to a totally different non kerberized HDFS cluster.
Each putHDFS is configured to use its own core-sites.xml. If I start the putHDFS that sends to the Kerberized HDFS cluster first, the other putHDFS will throw an error when it tries to send data to the non-kerberized HDFS cluster.
ERROR [Timer-Driven Process Thread-7] o.apache.nifi.processors.hadoop.PutHDFS
java.io.IOException: Failed on local exception: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections.; Host Details : local host is: "<client hostname>/<client IP>"; destination host is: "<hdfs hostname>":<hdfs port>
Caused by: java.io.IOException: Server asks us to fall back to SIMPLE auth, but this client is configured to only allow secure connections.
Even if i stop the putHDFS that is sending to the kerberized HDFS and/or restart the putHDFS sending to the non kerberized HDFS, the above error persists.
I need to restart NiFi to clear condition. After NiFi restart, if i run the putHDFS that sends to the non-kerberized HDFS first, the putHDFS to the kerberized HDFS will still work.