Camel
  1. Camel
  2. CAMEL-4017

Continue development of camel-twitter

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.10.0
    • Component/s: None
    • Labels:

      Description

      CAMEL-1520 resulted in the beginnings of camel-twitter, a component for consuming and publishing through the Twitter API. The issue is marked as WON'T fix, as the authors decide to start investigating a more generalized social media component.

      The src is currently sitting in Bruno's https://github.com/brunoborges/camel-twitter. I propose continuing development, hopefully adding it to the camel-extra repository.

      In my opinion, keeping this separate, rather than adding to a generalized social media plugin, would be important. Twitter usage, and the context the component would be used, can be drastically different.

      I would be more than willing to take ownership of the effort and begin to contribute. There's several areas of the Twitter API that we use that would be extremely helpful to add as a Camel endpoint (streaming connections, etc.).

      There are no Sub-Tasks for this issue.

        Activity

        Hide
        Brett Meyer added a comment -

        After discussing this with Bruno, we're going to re-startup development on camel-twitter, keeping it separate from camel-social. We'll be working on it at https://github.com/brunoborges/camel-twitter and hopefully releasing a stable build to camel-extra in the future. Contact us if interested in helping!

        Show
        Brett Meyer added a comment - After discussing this with Bruno, we're going to re-startup development on camel-twitter, keeping it separate from camel-social. We'll be working on it at https://github.com/brunoborges/camel-twitter and hopefully releasing a stable build to camel-extra in the future. Contact us if interested in helping!
        Hide
        Claus Ibsen added a comment -

        Brett that is really great. I would love to see a camel-twitter in the Camel distro soon.

        And the JARs are now in central maven repo, and they are Apache licensed.
        http://repo2.maven.org/maven2/org/twitter4j/

        So we should be able to include it at Apache out of the box.

        Show
        Claus Ibsen added a comment - Brett that is really great. I would love to see a camel-twitter in the Camel distro soon. And the JARs are now in central maven repo, and they are Apache licensed. http://repo2.maven.org/maven2/org/twitter4j/ So we should be able to include it at Apache out of the box.
        Hide
        Brett Meyer added a comment -

        Fantastic news! Is there any way to setup the Apache repository now, or should we continue to develop in GitHub and move it over later?

        Show
        Brett Meyer added a comment - Fantastic news! Is there any way to setup the Apache repository now, or should we continue to develop in GitHub and move it over later?
        Hide
        Claus Ibsen added a comment -

        Brett I suggest to continue at github, as its easy for you both to work together.

        So when you fell the component is ready to be donated to Apache. Then come back at this JIRA and we can take it from there.

        You may want to read this guide about how to write a component, as it have some good pointers
        http://camel.apache.org/add-new-component-guide.html

        Show
        Claus Ibsen added a comment - Brett I suggest to continue at github, as its easy for you both to work together. So when you fell the component is ready to be donated to Apache. Then come back at this JIRA and we can take it from there. You may want to read this guide about how to write a component, as it have some good pointers http://camel.apache.org/add-new-component-guide.html
        Hide
        Brett Meyer added a comment -

        Thanks Claus – will do!

        Show
        Brett Meyer added a comment - Thanks Claus – will do!
        Hide
        Brett Meyer added a comment -

        FYI, development has progressed significantly. I hope to be able to release a beta version within a few weeks.

        Show
        Brett Meyer added a comment - FYI, development has progressed significantly. I hope to be able to release a beta version within a few weeks.
        Hide
        Willem Jiang added a comment -

        @Brett,
        I'm interesting this twitter component, it could be great if we can put it into apache camel.

        Show
        Willem Jiang added a comment - @Brett, I'm interesting this twitter component, it could be great if we can put it into apache camel.
        Hide
        Michael Bevilacqua added a comment -

        @Brett
        I second Willem's comment!

        Show
        Michael Bevilacqua added a comment - @Brett I second Willem's comment!
        Hide
        Brett Meyer added a comment -

        All,

        We're definitely still working with it, but had to have a bit of a hiatus due to career changes. We'll keep you updated!

        Show
        Brett Meyer added a comment - All, We're definitely still working with it, but had to have a bit of a hiatus due to career changes. We'll keep you updated!
        Hide
        Hadrian Zbarcea added a comment -

        Actually there is quite a lot one could do with twitter with camel as it is today. I have some code almost done. I'll post it next week.

        Show
        Hadrian Zbarcea added a comment - Actually there is quite a lot one could do with twitter with camel as it is today. I have some code almost done. I'll post it next week.
        Hide
        Brett Meyer added a comment -

        All,

        The code has progressed significantly. I hope to release something within a few weeks. The plugin will be very comprehensive and exhaustive of most Twitter functionality.

        Show
        Brett Meyer added a comment - All, The code has progressed significantly. I hope to release something within a few weeks. The plugin will be very comprehensive and exhaustive of most Twitter functionality.
        Hide
        Willem Jiang added a comment -

        @Brett
        I checked out the camel-twitter component from the github https://github.com/brunoborges/camel-twitter. I'm already fixed some formate issues of the camel-twitter component, it should be ready to be committed into the camel trunk.
        I'm not sure if you have any other update of the component, if there is nothing new, I will commit the code shortly.

        Please let me what's your status.

        Willem

        Show
        Willem Jiang added a comment - @Brett I checked out the camel-twitter component from the github https://github.com/brunoborges/camel-twitter . I'm already fixed some formate issues of the camel-twitter component, it should be ready to be committed into the camel trunk. I'm not sure if you have any other update of the component, if there is nothing new, I will commit the code shortly. Please let me what's your status. Willem
        Hide
        Brett Meyer added a comment -

        Willem,

        Here's a few caveats:

        1.) I need to finish exhaustive testing. I'll complete this tonight.
        2.) The plugin encapsulates about 30% of the Twitter API, but most of the "core" areas are ready. I'm continuing to add new capabilities.
        3.) For the completed capabilities, they're all based on scheduled polling. I still need to finish direct route versions that will allow single-use, "get everything" functionality.
        4.) I need to put together some documentation on the endpoint URI usage.

        It's entirely your call as to when it should be committed to the trunk. If you'd like to proceed now, I'm more than happy to release the new work as patches throughout the next few weeks.

        Show
        Brett Meyer added a comment - Willem, Here's a few caveats: 1.) I need to finish exhaustive testing. I'll complete this tonight. 2.) The plugin encapsulates about 30% of the Twitter API, but most of the "core" areas are ready. I'm continuing to add new capabilities. 3.) For the completed capabilities, they're all based on scheduled polling. I still need to finish direct route versions that will allow single-use, "get everything" functionality. 4.) I need to put together some documentation on the endpoint URI usage. It's entirely your call as to when it should be committed to the trunk. If you'd like to proceed now, I'm more than happy to release the new work as patches throughout the next few weeks.
        Hide
        Brett Meyer added a comment -

        Willem,

        I was able to complete the code tonight. All of the "most useful" aspects of the Twitter API are included in our component. Both scheduled polling and direct routes are available for each.

        What do you need from us to proceed?

        There's certainly other areas of the API that I plan to include, but I can release those as patches later on.

        Show
        Brett Meyer added a comment - Willem, I was able to complete the code tonight. All of the "most useful" aspects of the Twitter API are included in our component. Both scheduled polling and direct routes are available for each. What do you need from us to proceed? There's certainly other areas of the API that I plan to include, but I can release those as patches later on.
        Hide
        Brett Meyer added a comment -

        Sorry guys, one more question. The unit tests rely on hard-coded tokens specific to a Twitter account. If the account's password is changed, the unit tests will fail. How should this be handled long-term in the trunk?

        Show
        Brett Meyer added a comment - Sorry guys, one more question. The unit tests rely on hard-coded tokens specific to a Twitter account. If the account's password is changed, the unit tests will fail. How should this be handled long-term in the trunk?
        Hide
        Willem Jiang added a comment -

        As the unit tests have the dependency of outside service, we normally don't run these tests in the CI box. I think we need to write an instruction to tell the user how to change the user and password configure when he wants to run the tests manually.

        Show
        Willem Jiang added a comment - As the unit tests have the dependency of outside service, we normally don't run these tests in the CI box. I think we need to write an instruction to tell the user how to change the user and password configure when he wants to run the tests manually.
        Hide
        Willem Jiang added a comment -

        @Brett,

        I'm planing to commit the code which I finished today, you can send me the new patches and I will merge them into the trunk.

        We could keep on enhancing the code in the trunk

        Willem

        Show
        Willem Jiang added a comment - @Brett, I'm planing to commit the code which I finished today, you can send me the new patches and I will merge them into the trunk. We could keep on enhancing the code in the trunk Willem
        Hide
        Brett Meyer added a comment -

        Willem,

        Tonight's changes were pretty substantial, so merging might be a pain. I apologize for continuing in github – I misread your comment from earlier and didn't realize you were already working on it in the trunk.

        Is it possible to use the current component in github as the starting point?

        Show
        Brett Meyer added a comment - Willem, Tonight's changes were pretty substantial, so merging might be a pain. I apologize for continuing in github – I misread your comment from earlier and didn't realize you were already working on it in the trunk. Is it possible to use the current component in github as the starting point?
        Hide
        Willem Jiang added a comment -

        Just one comment for the unit test of Twitter4JTest, I think it is just a simple test of Twitter4J and have nothing to do with the camel-twitter component. I will remove it from the camel-twitter unit test.

        Show
        Willem Jiang added a comment - Just one comment for the unit test of Twitter4JTest, I think it is just a simple test of Twitter4J and have nothing to do with the camel-twitter component. I will remove it from the camel-twitter unit test.
        Hide
        Willem Jiang added a comment -

        @Brett,
        It's OK, I will pull your change from the github.
        Once we put the code into trunk, we need to do the work which is based on apache trunk .

        Show
        Willem Jiang added a comment - @Brett, It's OK, I will pull your change from the github. Once we put the code into trunk, we need to do the work which is based on apache trunk .
        Hide
        Willem Jiang added a comment -

        @Brett,
        I just committed the camel-twitter component which is based on the last version of github.
        Here are the changes those I made:
        1. Reformatted the code, fixed some CS issues, remove the @author as we don't use it in camel
        2. Removed the MockProducer, TwitterUpdateStatusTest
        3. Added profile to run the twitter unit test, we don't run the twitter tests by default
        4. Changed the common log to slf4j which is used in camel by default
        5. Changed some info log to debug

        Show
        Willem Jiang added a comment - @Brett, I just committed the camel-twitter component which is based on the last version of github. Here are the changes those I made: 1. Reformatted the code, fixed some CS issues, remove the @author as we don't use it in camel 2. Removed the MockProducer, TwitterUpdateStatusTest 3. Added profile to run the twitter unit test, we don't run the twitter tests by default 4. Changed the common log to slf4j which is used in camel by default 5. Changed some info log to debug
        Hide
        Babak Vahdat added a comment -

        @Willem,

        could you please update the svn:ignore accordingly, see the attached screenshot for details.

        Show
        Babak Vahdat added a comment - @Willem, could you please update the svn:ignore accordingly, see the attached screenshot for details.
        Hide
        Brett Meyer added a comment -

        Willem, thank you! I appreciate the help.

        Is there anything I need to do to add it to Camel's components and features?

        I'll work on a wiki page tonight. Feel free to close this ticket. I'll open new ones with patches in the future. Thanks again.

        Show
        Brett Meyer added a comment - Willem, thank you! I appreciate the help. Is there anything I need to do to add it to Camel's components and features? I'll work on a wiki page tonight. Feel free to close this ticket. I'll open new ones with patches in the future. Thanks again.
        Hide
        Brett Meyer added a comment -

        @Willem

        Please see Babak's comment before this ticket is resolved.

        Also, do you need anything else from me to finish up the initial release? Do I need to do anything to OSGi-ready the component or create its feature? Or, did you guys already handle that?

        Thanks!

        Show
        Brett Meyer added a comment - @Willem Please see Babak's comment before this ticket is resolved. Also, do you need anything else from me to finish up the initial release? Do I need to do anything to OSGi-ready the component or create its feature? Or, did you guys already handle that? Thanks!
        Hide
        Willem Jiang added a comment -

        I update the svn:ignore property of the camel-twitter. I should be OK now.
        @Brett,
        Please update the wiki page if you have time, I will take care of camel-twitter feature.
        It looks like we need wrap the twitter jars in the servicemix as they are not osgi ready.

        Show
        Willem Jiang added a comment - I update the svn:ignore property of the camel-twitter. I should be OK now. @Brett, Please update the wiki page if you have time, I will take care of camel-twitter feature. It looks like we need wrap the twitter jars in the servicemix as they are not osgi ready.
        Hide
        Claus Ibsen added a comment -

        I polished the code a bit.

        There is a number of TODO in the source which needs to be looked at.
        Also instead of custom initialize method, use the doStart / doStop methods from Camel to do init / cleanup logic.

        Show
        Claus Ibsen added a comment - I polished the code a bit. There is a number of TODO in the source which needs to be looked at. Also instead of custom initialize method, use the doStart / doStop methods from Camel to do init / cleanup logic.
        Hide
        Brett Meyer added a comment -

        @Willem

        I'll work on the wiki ASAP. SMX4-1032 updated org.apache.servicemix.bundles.twitter4j to the newest Twitter4J jars. camel-twitter dependencies and features should probably be updated to use them.

        @Claus,

        Thanks for working with it! There's definitely a few TODOs that I'll keep working with.

        Show
        Brett Meyer added a comment - @Willem I'll work on the wiki ASAP. SMX4-1032 updated org.apache.servicemix.bundles.twitter4j to the newest Twitter4J jars. camel-twitter dependencies and features should probably be updated to use them. @Claus, Thanks for working with it! There's definitely a few TODOs that I'll keep working with.
        Hide
        Brett Meyer added a comment -

        I'm resolving this ticket. I see that twitter4j-bundle-version 2.2.5_1-SNAPSHOT was added to the features. The component should be all set. I'll release TODO fixes, new features, etc. under new tickets.

        Thanks for the help Willem and Claus!

        Show
        Brett Meyer added a comment - I'm resolving this ticket. I see that twitter4j-bundle-version 2.2.5_1-SNAPSHOT was added to the features. The component should be all set. I'll release TODO fixes, new features, etc. under new tickets. Thanks for the help Willem and Claus!

          People

          • Assignee:
            Unassigned
            Reporter:
            Brett Meyer
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development