Camel
  1. Camel
  2. CAMEL-3471

A new Apache Mina 2.0.x component

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.5.0
    • Fix Version/s: 2.10.0
    • Component/s: camel-mina
    • Labels:
      None

      Description

      Apache Mina 2.0.3 is out: http://mina.apache.org/downloads.html

      1. MinaUdpNoCamelTest.java
        5 kB
        Chad Beaulac
      2. CAMEL-3471-all-tests-pass.patch
        296 kB
        Chad Beaulac
      3. CAMEL-3471.patch
        53 kB
        Christian Müller

        Issue Links

        There are no Sub-Tasks for this issue.

          Activity

          Hide
          Chad Beaulac added a comment -

          Patch has been applied to SVN HEAD. Created subtask to add the wiki page for the new component. It isn't assigned to anybody yet.

          Show
          Chad Beaulac added a comment - Patch has been applied to SVN HEAD. Created subtask to add the wiki page for the new component. It isn't assigned to anybody yet.
          Hide
          Claus Ibsen added a comment -

          Okay I got the osgi imports fixed so camel-mina and camel-mina2 component have the correct version ranges.

          Show
          Claus Ibsen added a comment - Okay I got the osgi imports fixed so camel-mina and camel-mina2 component have the correct version ranges.
          Hide
          Claus Ibsen added a comment -

          Thanks for the patch.

          I have applied it to trunk. I polished a few minor issues such as removing some not needed logging, as well logging to a file during testing.
          The Apache Mina 2 JARs is already OSGi bundles, so no need for a SMX wrapper.

          We will need though to ensure the imports of camel-mina is limited to 1.x. Where as camel-mina2 should be 2.x+. Will look into this later.

          Show
          Claus Ibsen added a comment - Thanks for the patch. I have applied it to trunk. I polished a few minor issues such as removing some not needed logging, as well logging to a file during testing. The Apache Mina 2 JARs is already OSGi bundles, so no need for a SMX wrapper. We will need though to ensure the imports of camel-mina is limited to 1.x. Where as camel-mina2 should be 2.x+. Will look into this later.
          Hide
          Chad Beaulac added a comment -

          All tests pass in this patch. The port works like the Mina-1 endpoint using CountDownLatch to block for responses.
          Checkstyle build passes.

          Show
          Chad Beaulac added a comment - All tests pass in this patch. The port works like the Mina-1 endpoint using CountDownLatch to block for responses. Checkstyle build passes.
          Hide
          Chad Beaulac added a comment -

          I need to do the following before providing the patch

          • Merge with upstream changes (in progress)
          • Clean build and resolve any issues
          • Build with checkstyle and resolve any issues

          I have not added a wiki page for this new component yet. So, we'll have to do that too.

          The asynchronous use cases could modify some behavior significantly as the current implementation uses blocking I/O with CountDownLatch like the mina 1 component. I was thinking of providing a Mina2 IoHandlerAdapter to the Mina2Endpoint to be able to send "n" messages and receive "m" messages asynchronously on a single TCP stream.

          Show
          Chad Beaulac added a comment - I need to do the following before providing the patch Merge with upstream changes (in progress) Clean build and resolve any issues Build with checkstyle and resolve any issues I have not added a wiki page for this new component yet. So, we'll have to do that too. The asynchronous use cases could modify some behavior significantly as the current implementation uses blocking I/O with CountDownLatch like the mina 1 component. I was thinking of providing a Mina2 IoHandlerAdapter to the Mina2Endpoint to be able to send "n" messages and receive "m" messages asynchronously on a single TCP stream.
          Hide
          Chad Beaulac added a comment -

          No problem. We can resolve 3471 first.

          Show
          Chad Beaulac added a comment - No problem. We can resolve 3471 first.
          Hide
          Claus Ibsen added a comment -

          Chad could we get this ticket resolved first, eg a camel-mina2 component that is similar to the existing. Then after this focus on those 2 other JIRA tickets. Then its easier to track.

          So if you can attach a patch what you got now would be great.

          Show
          Claus Ibsen added a comment - Chad could we get this ticket resolved first, eg a camel-mina2 component that is similar to the existing. Then after this focus on those 2 other JIRA tickets. Then its easier to track. So if you can attach a patch what you got now would be great.
          Hide
          Christian Müller added a comment -

          I am pleased to see some progress here.

          Show
          Christian Müller added a comment - I am pleased to see some progress here.
          Hide
          Chad Beaulac added a comment -

          Finally. All 81 unit tests pass. Now I'm working on adding unit tests for CAMEL-1075 and CAMEL-2624 because that's the functionality I need.

          Show
          Chad Beaulac added a comment - Finally. All 81 unit tests pass. Now I'm working on adding unit tests for CAMEL-1075 and CAMEL-2624 because that's the functionality I need.
          Hide
          Chad Beaulac added a comment -

          Hi Thomas. There are currently 4 unit tests failing for the mina2 component. The failures are unrelated to the original UDP issues. I'll work to get them resolved and post a patch. This by itself does not help with CAMEL-1075 and CAMEL-2624 for asynchronous message processing which is what I'm most interested in. The java.nio framework lends itself to asynchronous processing. The TemplateProducer and mina(1) component are geared toward synchronous request-reply processing. So, I've been wondering if we should support all of the synchronous use cases that mina(1) does like using the template.requestBody methods or if mina2 should move to full asynchronous processing only? A full asynchronous implementation will be easier but it does not provide a smooth transition for mina(1) users. Receiving a response would be done asynchronously in a Mina2 IoHandlerAdapter. I've been thinking about how to inject an IoHandlerAdapter into the Mina2Producer when using TemplateProducer. I'll have a patch soon. It's a straight port of mina(1) and doesn't cover the Jira tickets referenced. Finishing CAMEL-1075 and CAMEL-2624 is a top priority of mine as soon as this ticket is resolved.

          Show
          Chad Beaulac added a comment - Hi Thomas. There are currently 4 unit tests failing for the mina2 component. The failures are unrelated to the original UDP issues. I'll work to get them resolved and post a patch. This by itself does not help with CAMEL-1075 and CAMEL-2624 for asynchronous message processing which is what I'm most interested in. The java.nio framework lends itself to asynchronous processing. The TemplateProducer and mina(1) component are geared toward synchronous request-reply processing. So, I've been wondering if we should support all of the synchronous use cases that mina(1) does like using the template.requestBody methods or if mina2 should move to full asynchronous processing only? A full asynchronous implementation will be easier but it does not provide a smooth transition for mina(1) users. Receiving a response would be done asynchronously in a Mina2 IoHandlerAdapter. I've been thinking about how to inject an IoHandlerAdapter into the Mina2Producer when using TemplateProducer. I'll have a patch soon. It's a straight port of mina(1) and doesn't cover the Jira tickets referenced. Finishing CAMEL-1075 and CAMEL-2624 is a top priority of mine as soon as this ticket is resolved.
          Hide
          Thomas Johansen added a comment -

          When can I expect a Camel release which includes this component? And in the mean time, will the component be accessible from some other place, and be compatible with Camel 2.8/2.9?

          Show
          Thomas Johansen added a comment - When can I expect a Camel release which includes this component? And in the mean time, will the component be accessible from some other place, and be compatible with Camel 2.8/2.9?
          Hide
          Chad Beaulac added a comment -

          Added a new camel-mina2 component to my Git-svn sandbox.
          Finally tracked down the UDP issue to the TypeConverter!

          Show
          Chad Beaulac added a comment - Added a new camel-mina2 component to my Git-svn sandbox. Finally tracked down the UDP issue to the TypeConverter!
          Hide
          Chad Beaulac added a comment -

          Excellent idea. Much less breakage. I'll update my Git repo from svn HEAD and add the mina2 component to it.

          Show
          Chad Beaulac added a comment - Excellent idea. Much less breakage. I'll update my Git repo from svn HEAD and add the mina2 component to it.
          Hide
          Claus Ibsen added a comment -

          We should create a new camel-mina2 component. Then we can keep the old camel-mina for Mina 1.x end users.

          Show
          Claus Ibsen added a comment - We should create a new camel-mina2 component. Then we can keep the old camel-mina for Mina 1.x end users.
          Hide
          Chad Beaulac added a comment -

          I added this unit test to my git-svn Camel sandbox. It tests that we can setup a Mina UDP server and client and send multiple messages. This indicates that the UDP problems in camel-mina are in the camel-mina Component impl.

          Show
          Chad Beaulac added a comment - I added this unit test to my git-svn Camel sandbox. It tests that we can setup a Mina UDP server and client and send multiple messages. This indicates that the UDP problems in camel-mina are in the camel-mina Component impl.
          Hide
          Chad Beaulac added a comment -

          The UDP endpoint works when you have a codec configured. If not, you receive messages over and over until you run out of RAM and the test dies.

          Show
          Chad Beaulac added a comment - The UDP endpoint works when you have a codec configured. If not, you receive messages over and over until you run out of RAM and the test dies.
          Hide
          Chad Beaulac added a comment - - edited

          Updated camel-quickfix to use Mina 1.1.7. This overrides the global Mina version of 2.x. Updated camel-hl7 codec to use the Mina 2.x API. Removed slf4j-api excludes from camel-hl7 so it could resolve dependencies. Added aQute repository to camel-itest-osgi pom. All source and tests in my sandbox compile now.
          (I'd log this in the work log but it seems I don't have that permission)

          Show
          Chad Beaulac added a comment - - edited Updated camel-quickfix to use Mina 1.1.7. This overrides the global Mina version of 2.x. Updated camel-hl7 codec to use the Mina 2.x API. Removed slf4j-api excludes from camel-hl7 so it could resolve dependencies. Added aQute repository to camel-itest-osgi pom. All source and tests in my sandbox compile now. (I'd log this in the work log but it seems I don't have that permission)
          Hide
          Chad Beaulac added a comment - - edited

          I'm working through Camel maven build dependencies. I'm in the process of adding a Mina UDP no-Camel test to indicate where the UDP test problems reside. I've hit another issue in the dependencies: camel-quickfix depends on QuickFixJ 1.5.0 which uses Mina 1.0.x. How can I update Mina in Camel to 2.0.x and support the camel-quickfix component at the same time? Should I put a mina-core dependency for 1.0.x in camel-quickfix?

          Ok, that's what I did. I put a dependency in camel-quickfix for mina 1.1.7.

          Show
          Chad Beaulac added a comment - - edited I'm working through Camel maven build dependencies. I'm in the process of adding a Mina UDP no-Camel test to indicate where the UDP test problems reside. I've hit another issue in the dependencies: camel-quickfix depends on QuickFixJ 1.5.0 which uses Mina 1.0.x. How can I update Mina in Camel to 2.0.x and support the camel-quickfix component at the same time? Should I put a mina-core dependency for 1.0.x in camel-quickfix? Ok, that's what I did. I put a dependency in camel-quickfix for mina 1.1.7.
          Hide
          Chad Beaulac added a comment -

          Modified description. Apache Mina 2.0.3 is out.

          Show
          Chad Beaulac added a comment - Modified description. Apache Mina 2.0.3 is out.
          Hide
          Claus Ibsen added a comment - - edited

          Chad I upgraded your account so you can assign tickets.

          Show
          Claus Ibsen added a comment - - edited Chad I upgraded your account so you can assign tickets.
          Hide
          Chad Beaulac added a comment -

          Hi Claus. Updating the hl7 codec shouldn't be a problem. You want to assign it to me and I'll give it a whack? Seems like I don't have that permission on your Jira instance.

          Show
          Chad Beaulac added a comment - Hi Claus. Updating the hl7 codec shouldn't be a problem. You want to assign it to me and I'll give it a whack? Seems like I don't have that permission on your Jira instance.
          Hide
          Claus Ibsen added a comment -

          Thanks Chad. There is also the camel-hl7 component which is a mina codec. It may also need an API upgrade.

          Show
          Claus Ibsen added a comment - Thanks Chad. There is also the camel-hl7 component which is a mina codec. It may also need an API upgrade.
          Hide
          Chad Beaulac added a comment -

          I could do this and provide a Mina codec for Google Protocol Buffers. I developed a protobuf codec for Mina 1.x and 2.x.

          Show
          Chad Beaulac added a comment - I could do this and provide a Mina codec for Google Protocol Buffers. I developed a protobuf codec for Mina 1.x and 2.x.
          Hide
          Christian Müller added a comment -

          I will give up this issue to someone who has more experience with Apache Mina because I couldn't solve the issue.

          Show
          Christian Müller added a comment - I will give up this issue to someone who has more experience with Apache Mina because I couldn't solve the issue.
          Hide
          Hadrian Zbarcea added a comment -

          Just a reminder (probably useless). Please don't commit such non trivial patches until the camel-2.7.0 vote passes.

          Show
          Hadrian Zbarcea added a comment - Just a reminder (probably useless). Please don't commit such non trivial patches until the camel-2.7.0 vote passes.
          Hide
          Christian Müller added a comment -

          I attached my latest patch, which unfortunately doesn't work to 100%:

          I think the issues are in Mina, but I do not have any prove. But it's very strange, when I send 3 messages and I receive 13000 messages...
          I will posted this on the Mina user list to get help from the Mina guys. I hope someone else who is more familiar with Mina can help out!?

          [1] http://old.nabble.com/Problems-by-upgrading-the-Apache-Camel-Mina-component-from-Mina-1.1.7-to-2.0.2-td31175927.html

          Christian

          Show
          Christian Müller added a comment - I attached my latest patch, which unfortunately doesn't work to 100%: I think the issues are in Mina, but I do not have any prove. But it's very strange, when I send 3 messages and I receive 13000 messages... I will posted this on the Mina user list to get help from the Mina guys. I hope someone else who is more familiar with Mina can help out!? [1] http://old.nabble.com/Problems-by-upgrading-the-Apache-Camel-Mina-component-from-Mina-1.1.7-to-2.0.2-td31175927.html Christian
          Hide
          Christian Müller added a comment -

          Thomas,
          I attached my current status of work (diff against trunk). I think I updated all pieces of the code correct, but some of the tests didn't work.
          At present, 6 test of 82 fails or has errors. I don't know what the problems are. I will have a second look in the next days...

          You are more then welcome to help us out to solve this issue, if you want. We love contributions...

          Cheers,
          Christian

          Show
          Christian Müller added a comment - Thomas, I attached my current status of work (diff against trunk). I think I updated all pieces of the code correct, but some of the tests didn't work. At present, 6 test of 82 fails or has errors. I don't know what the problems are. I will have a second look in the next days... You are more then welcome to help us out to solve this issue, if you want. We love contributions... Cheers, Christian
          Hide
          Thomas Johansen added a comment -

          Hi Christian, I'm already using Camel 2.6. I suspect that a camel-mina using Mina 2 would simplify my throttling implementation. Currently (using Camel 2.6) I have to:

          1. Extend MinaComponent with an overridden createSocketEndpoint(String,MinaConfiguration), which is a copy/paste where I'm using MySocketAcceptor instead of SocketAcceptor and have disconnectOnUnbind on acceptorConfig configurable.
          2. Extend SocketAcceptor with methods bind() and unbind() (which use cached parameters from previous bind(...) and unbind(...)).

          Not pretty...

          So I understand camel-mina with Mina 2 will arrive with Camel 2.8. I was really wondering if a working camel-mina-2.8 beta was released shortly, and if I could use that with camel-core-2.6?

          Cheers

          Show
          Thomas Johansen added a comment - Hi Christian, I'm already using Camel 2.6. I suspect that a camel-mina using Mina 2 would simplify my throttling implementation. Currently (using Camel 2.6) I have to: 1. Extend MinaComponent with an overridden createSocketEndpoint(String,MinaConfiguration), which is a copy/paste where I'm using MySocketAcceptor instead of SocketAcceptor and have disconnectOnUnbind on acceptorConfig configurable. 2. Extend SocketAcceptor with methods bind() and unbind() (which use cached parameters from previous bind(...) and unbind(...)). Not pretty... So I understand camel-mina with Mina 2 will arrive with Camel 2.8. I was really wondering if a working camel-mina-2.8 beta was released shortly, and if I could use that with camel-core-2.6? Cheers
          Hide
          Christian Müller added a comment -

          Hello Thomas!
          Camel 2.6 is already out and Camel 2.7 will be released this month (we already have a feature freeze). This issue is actual target for Camel 2.8.
          Cheers,
          Christian

          Show
          Christian Müller added a comment - Hello Thomas! Camel 2.6 is already out and Camel 2.7 will be released this month (we already have a feature freeze). This issue is actual target for Camel 2.8. Cheers, Christian
          Hide
          Thomas Johansen added a comment -

          What's the status for this issue now? I would like to use Mina 2 with Camel 2.6.0, because I need simple connection throttling (using unbind/bind with closeOnDeactivation=false through an IoFilter).

          Show
          Thomas Johansen added a comment - What's the status for this issue now? I would like to use Mina 2 with Camel 2.6.0, because I need simple connection throttling (using unbind/bind with closeOnDeactivation=false through an IoFilter).
          Hide
          Claus Ibsen added a comment -

          Walter that's another story, that would be best to discuss at the mailing list, so more people can get involved.

          You would need a strategy for how to correlate the replies, which must be part of the protocol. So the end user must have a way of telling the camel-mina how it should correlate replies with the request messages.

          And the camel-mina component need to have logic where it stackup requests, and accepts incoming replies, and correlate those. And it may have to handle timeout, and unknown replies coming back etc. All kinda of logic we already have in the camel-jms, so its possible.

          Show
          Claus Ibsen added a comment - Walter that's another story, that would be best to discuss at the mailing list, so more people can get involved. You would need a strategy for how to correlate the replies, which must be part of the protocol. So the end user must have a way of telling the camel-mina how it should correlate replies with the request messages. And the camel-mina component need to have logic where it stackup requests, and accepts incoming replies, and correlate those. And it may have to handle timeout, and unknown replies coming back etc. All kinda of logic we already have in the camel-jms, so its possible.
          Hide
          Walter Chang added a comment -

          Can we expect non-blocking(async) InOut support as well?

          Show
          Walter Chang added a comment - Can we expect non-blocking(async) InOut support as well?
          Hide
          Christian Müller added a comment -

          Jeffrey,
          if you could add a diff to this issue, I could have a look on the code.
          I think I solved these issues, but got the "java.net.BindException: Address already in use" exception.
          Maybe we can solve this issue together...

          Christian

          Show
          Christian Müller added a comment - Jeffrey, if you could add a diff to this issue, I could have a look on the code. I think I solved these issues, but got the "java.net.BindException: Address already in use" exception. Maybe we can solve this issue together... Christian
          Hide
          Jeffrey Knight added a comment -

          I do not get the BindException. Most of the exceptions I'm getting are around connections not being in the correct state. I'll put some more time into it and will hopefully get it up and running.

          Mina 2 removed "ThreadModel" which is used widely by camel-mina, so I'm trying to get this integrated back in:
          "In 2.x, you have to add an ExecutorFilter by yourself explicitly, of course only when you want to add it." [1]

          [1] http://mina.apache.org/changes-between-2x-and-1x.html#ChangesBetween2.xand1.x-%257B%257BIoService%257D%257Dconfigurationhasbeensimplified.

          Show
          Jeffrey Knight added a comment - I do not get the BindException. Most of the exceptions I'm getting are around connections not being in the correct state. I'll put some more time into it and will hopefully get it up and running. Mina 2 removed "ThreadModel" which is used widely by camel-mina, so I'm trying to get this integrated back in: "In 2.x, you have to add an ExecutorFilter by yourself explicitly, of course only when you want to add it." [1] [1] http://mina.apache.org/changes-between-2x-and-1x.html#ChangesBetween2.xand1.x-%257B%257BIoService%257D%257Dconfigurationhasbeensimplified .
          Hide
          Christian Müller added a comment -

          Jeff,
          we love contributions. If you have some work done and you would like to share it with us, you are welcome.

          Attach the diff (actual TRUNK and your code) to this issue and check the "Grant license to ASF for inclusion in ASF works (as per the Apache License §5)" box. You also have to fill out the Individual Contributor License Agreement and send it to us (you will find the details in there).

          In your mail you wrote your code compile but some of the unit tests fails. Do you also get the "java.net.BindException: Address already in use"? May be this is a (known) Mina issue? Or we do something wrong when we close our connections? Do you have time and pleasure to investigate this issue?

          Christian

          Show
          Christian Müller added a comment - Jeff, we love contributions. If you have some work done and you would like to share it with us, you are welcome. Attach the diff (actual TRUNK and your code) to this issue and check the "Grant license to ASF for inclusion in ASF works (as per the Apache License §5)" box. You also have to fill out the Individual Contributor License Agreement and send it to us (you will find the details in there). In your mail you wrote your code compile but some of the unit tests fails. Do you also get the "java.net.BindException: Address already in use"? May be this is a (known) Mina issue? Or we do something wrong when we close our connections? Do you have time and pleasure to investigate this issue? Christian
          Hide
          Jeffrey Knight added a comment - - edited

          One example: IoFilter was reoganized to a new package:

          • mina-core-1.1.7.jar:/org/apache/mina/common/IoFilter.class
          • mina-core-2.0.2.jar:/org/apache/mina/core/filterchain/IoFilter.class

          Mina 1x vs 2x:

          Show
          Jeffrey Knight added a comment - - edited One example: IoFilter was reoganized to a new package: mina-core-1.1.7.jar:/org/apache/mina/common/IoFilter.class mina-core-2.0.2.jar:/org/apache/mina/core/filterchain/IoFilter.class Mina 1x vs 2x: http://mina.apache.org/changes-between-2x-and-1x.html
          Hide
          Christian Müller added a comment -

          Because of the major release upgrade, some code changes were necessary. At present my code compiles, but a few test fails with "java.net.BindException: Address already in use" exception. I have to spend more time on this issue.
          I have also to check, whether all current Mina 1.x options are still supported in Mina 2.x.

          Show
          Christian Müller added a comment - Because of the major release upgrade, some code changes were necessary. At present my code compiles, but a few test fails with "java.net.BindException: Address already in use" exception. I have to spend more time on this issue. I have also to check, whether all current Mina 1.x options are still supported in Mina 2.x.

            People

            • Assignee:
              Chad Beaulac
              Reporter:
              Christian Müller
            • Votes:
              4 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development