[NIFI-7997] ListFTP does not use HTTP proxy

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.11.4
• Component/s: Extensions
• Assignee: David Handermann
• Reporter: Richard Zuidhof
• Created: 11/Nov/20 12:05
• Updated: 12/Jan/22 14:23
• Resolved: 12/Jan/22 14:23

Description

After configuring a HTTP type proxy server in the ListFTP processor Nifi still tries to connect directly to the FTP server instead of the proxy server for the initial connection. Subsequent commands like DIR or GET are tunneled through the HTTP proxy but without a direct connection to the FTP server this will never succeeded. There is no issue using a SOCKS proxy.

You can reproduce this very easily, even without a real proxy. Just enter a non existing ftp host and set proxy type to HTTP and use a fake proxy host and port. When using a non-existing ftp hostname you will get this error. This shows Nifi wants to connect directly instead of via the proxy:

2020-11-11 12:36:01,002 ERROR [Timer-Driven Process Thread-8] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to nowhere.to.go: java.net.UnknownHostException: nowhere.to.go

When using an unreachable IP (like 10.123.1.1) you will see this:

2020-11-11 12:43:10,862 ERROR [Timer-Driven Process Thread-10] o.a.nifi.processors.standard.ListFTP ListFTP[id=99240cfd-0175-1000-ffff-ffffe04e112b] Failed to perform listing on remote host due to Connection timed out (Connection timed out): java.net.ConnectException: Connection timed out (Connection timed out) java.net.ConnectException: Connection timed out (Connection timed out) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:607) at org.apache.commons.net.SocketClient._connect(SocketClient.java:243) at org.apache.commons.net.SocketClient.connect(SocketClient.java:181) at org.apache.nifi.processors.standard.util.FTPTransfer.getClient(FTPTransfer.java:600) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:233) at org.apache.nifi.processors.standard.util.FTPTransfer.getListing(FTPTransfer.java:196) at org.apache.nifi.processors.standard.ListFileTransfer.performListing(ListFileTransfer.java:106) at org.apache.nifi.processor.util.list.AbstractListProcessor.listByTrackingTimestamps(AbstractListProcessor.java:472) at org.apache.nifi.processor.util.list.AbstractListProcessor.onTrigger(AbstractListProcessor.java:414) at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176) at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213) at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117) at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Issue Links

• duplicates: NIFI-6266 FTP processors do not honor Proxy settings