Camel
  1. Camel
  2. CAMEL-3655

Add FtpPollingConsumer to better support polling consumer EIPs

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0
    • Component/s: camel-ftp
    • Labels:
      None

      Description

      See nabble
      http://camel.465427.n5.nabble.com/Event-driven-FTP-consumer-tp3376242p3376242.html

      The PollingConsumer should only poll once and thus it should not be schedule based, and it should disconnect after polling.

        Activity

        Hide
        Richard Kettelerij added a comment - - edited

        @Claus,

        Thanks for creating the ticket (was about to do it myself but you were quicker ). Anyway the issue I faced with the FTP component was that there's no option to read files on-demand from FTP (or plain filesystem) in the middle of a route. The only way to read files is to use a continuously polling consumer, which is fine it a lot of case I guess.

        Now this can be solved by having a consumer that polls once and disconnects so you can use it a content enricher. This should allow folks to download multiple files in one poll and continue the route. Hadrian suggested using it in a pipeline (using to()) but this requires a producer that can read from FTP instead of a consumer (CAMEL-3397).

        Also this might be a more general issue, shouldn't all polling consumers allow for this behavior? It certainly also affects the file: component.

        Show
        Richard Kettelerij added a comment - - edited @Claus, Thanks for creating the ticket (was about to do it myself but you were quicker ). Anyway the issue I faced with the FTP component was that there's no option to read files on-demand from FTP (or plain filesystem) in the middle of a route. The only way to read files is to use a continuously polling consumer, which is fine it a lot of case I guess. Now this can be solved by having a consumer that polls once and disconnects so you can use it a content enricher. This should allow folks to download multiple files in one poll and continue the route. Hadrian suggested using it in a pipeline (using to() ) but this requires a producer that can read from FTP instead of a consumer ( CAMEL-3397 ). Also this might be a more general issue, shouldn't all polling consumers allow for this behavior? It certainly also affects the file: component.
        Hide
        Claus Ibsen added a comment -

        We may want to require that pollEnrich with file/ftp requires a file name to be specified? It seems a bit arbitrary to enrich a message with whatever the file/ftp consumer can scan and find in the directories.

        So I most likely add that as a requirement and check that a fileName is specified and fail if not.

        Show
        Claus Ibsen added a comment - We may want to require that pollEnrich with file/ftp requires a file name to be specified? It seems a bit arbitrary to enrich a message with whatever the file/ftp consumer can scan and find in the directories. So I most likely add that as a requirement and check that a fileName is specified and fail if not.
        Hide
        Claus Ibsen added a comment -

        Ah I spotted likely an easy solution which is to resume/suspend the consumer upon demand. Then the ftp consumer should not keep polling as its suspended, if not in demand.

        I will test that and see if we can use that as a workaround for Camel 2.8. And then talk about a solution for Camel 2.9 where we develop a dedicated polling consumer for file/ftp that isn't scheduled based.

        Show
        Claus Ibsen added a comment - Ah I spotted likely an easy solution which is to resume/suspend the consumer upon demand. Then the ftp consumer should not keep polling as its suspended, if not in demand. I will test that and see if we can use that as a workaround for Camel 2.8. And then talk about a solution for Camel 2.9 where we develop a dedicated polling consumer for file/ftp that isn't scheduled based.
        Hide
        Claus Ibsen added a comment -

        I found a better solution for Camel 2.8. So now it should work with FTP.

        Show
        Claus Ibsen added a comment - I found a better solution for Camel 2.8. So now it should work with FTP.
        Hide
        Richard Kettelerij added a comment - - edited

        Nice work Claus. Seems like quite a change, I'll try testing it out when I get the chance.

        Show
        Richard Kettelerij added a comment - - edited Nice work Claus. Seems like quite a change, I'll try testing it out when I get the chance.
        Hide
        Luc-André Essiambre added a comment -

        Is it possible to know what the solution is? Because I can't get this thing to work. I basically have to download a file when receiving a JMS message.

        Show
        Luc-André Essiambre added a comment - Is it possible to know what the solution is? Because I can't get this thing to work. I basically have to download a file when receiving a JMS message.

          People

          • Assignee:
            Claus Ibsen
            Reporter:
            Claus Ibsen
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development