Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      New JMSAdaptor and supporting classes to consume from a JMS topic.

      Description

      We should have a JMSAdaptor that listens to a JMS queue for messages to send to a collector.

        Activity

        Hide
        billgraham Bill Graham added a comment -

        Please comment with any suggestions. This is what I'm thinking:

        • oah.chukwa.datacollection.adaptor.jms
          This is the package where the code will live. Creating a new package since there will be a couple of related classes.
        • JMSAdaptor
          This is the main class that will connect to a JMS topic, listen for messages and add chunks to the receiver. It uses a JMSMessageTransformer to transform the message to a byte array.
        • JMSMessageTransformer
          An interface that knows how to transform a JMSMessage into a byte array with the following method:
          public byte[] transform(javax.jms.Message message)
        • JMSTextMessageTransformer
          This will be the default transformer, which transforms a javax.jms.TextMessage's payload into a byte array
        • JMSMessagePropertyTransformer
          This is what I'd personally need so I might add this as well. It takes a collection of ordered message property names and a delimiter and returns a byte array made using the corresponding property values.
        • Configuration would look something like this:
          add JMSAdaptor <dataType> <brokerURL> -t <topicName> [-s <JMSSelector>] [-x <transformerName>] [-p <transformerProperties>] <offset>
        • New dependancies
          Sun's JMS
          Sun's J2EE Management
          Apache ActiveMQ
          Apache Commons Pool

        This first pass would only work with a topic but subsequent releases could work with a queue if the need arises. The selector is to filter on certain types of messages. TransformerName is to override the default transformer and transformerProperties is whatever properties it takes.

        Show
        billgraham Bill Graham added a comment - Please comment with any suggestions. This is what I'm thinking: oah.chukwa.datacollection.adaptor.jms This is the package where the code will live. Creating a new package since there will be a couple of related classes. JMSAdaptor This is the main class that will connect to a JMS topic, listen for messages and add chunks to the receiver. It uses a JMSMessageTransformer to transform the message to a byte array. JMSMessageTransformer An interface that knows how to transform a JMSMessage into a byte array with the following method: public byte[] transform(javax.jms.Message message) JMSTextMessageTransformer This will be the default transformer, which transforms a javax.jms.TextMessage 's payload into a byte array JMSMessagePropertyTransformer This is what I'd personally need so I might add this as well. It takes a collection of ordered message property names and a delimiter and returns a byte array made using the corresponding property values. Configuration would look something like this: add JMSAdaptor <dataType> <brokerURL> -t <topicName> [-s <JMSSelector>] [-x <transformerName>] [-p <transformerProperties>] <offset> New dependancies Sun's JMS Sun's J2EE Management Apache ActiveMQ Apache Commons Pool This first pass would only work with a topic but subsequent releases could work with a queue if the need arises. The selector is to filter on certain types of messages. TransformerName is to override the default transformer and transformerProperties is whatever properties it takes.
        Hide
        jboulon Jerome Boulon added a comment -

        Hi,
        Can we have a better partitioning for jars files? I would like to get jar requires at compile time and another set requires at runtime depending on what is really needed. Because this could become a compatibility issue pretty fast as people may use different versions and would not have any good reason to update/downgrade their jars if they are not using Apache ActiveMQ for example.
        This could be achieve using ivy module configuration.

        Show
        jboulon Jerome Boulon added a comment - Hi, Can we have a better partitioning for jars files? I would like to get jar requires at compile time and another set requires at runtime depending on what is really needed. Because this could become a compatibility issue pretty fast as people may use different versions and would not have any good reason to update/downgrade their jars if they are not using Apache ActiveMQ for example. This could be achieve using ivy module configuration.
        Hide
        billgraham Bill Graham added a comment -

        I'm all for what you're suggesting, but I think that should be tackled in a separate bug by someone with more ivy experience than myself.

        The ActiveMQ jar (or any JMS implementation library for that matter) will be needed at runtime to get a ConnectionFactory. This doesn't doesn't bind the user into running an AMQ broker though, it just says that the JMSAdaptor is going to use the AMQ impl to establish it's connections.

        Show
        billgraham Bill Graham added a comment - I'm all for what you're suggesting, but I think that should be tackled in a separate bug by someone with more ivy experience than myself. The ActiveMQ jar (or any JMS implementation library for that matter) will be needed at runtime to get a ConnectionFactory. This doesn't doesn't bind the user into running an AMQ broker though, it just says that the JMSAdaptor is going to use the AMQ impl to establish it's connections.
        Hide
        eyang Eric Yang added a comment -

        The design looks good.

        Show
        eyang Eric Yang added a comment - The design looks good.
        Hide
        billgraham Bill Graham added a comment -

        Attaching CHUKWA-469.1.patch, which includes JMSAdaptor and supporting classes as described above. There are many unit tests to look at for usage of this adaptor. Once it's incorporated I'll can of course add documentation to the site as well.

        My previous comment above was incorrect though, as I believe this implementation is bound to having an ActiveMQ broker. I've introduced one dependency into ivy, which is activemq-core. I've also tried to isolate the dependency so it could be subclassed and swapped out. In JMSAdaptor this method is the one place where AMQ is integrated:

         
        protected ConnectionFactory initializeConnectionFactory(String brokerURL) {
            return new ActiveMQConnectionFactory(brokerURL);
        }
        

        Any suggestions for how to best handle this dependency? We could say that JMSAdaptor by default uses AMQ, but it could be subclassed for other JMS providers. Or we could make JMSAdaptor abstact and have a concrete ActiveMQJMSAdaptor subclass.

        Show
        billgraham Bill Graham added a comment - Attaching CHUKWA-469 .1.patch, which includes JMSAdaptor and supporting classes as described above. There are many unit tests to look at for usage of this adaptor. Once it's incorporated I'll can of course add documentation to the site as well. My previous comment above was incorrect though, as I believe this implementation is bound to having an ActiveMQ broker. I've introduced one dependency into ivy, which is activemq-core. I've also tried to isolate the dependency so it could be subclassed and swapped out. In JMSAdaptor this method is the one place where AMQ is integrated: protected ConnectionFactory initializeConnectionFactory(String brokerURL) { return new ActiveMQConnectionFactory(brokerURL); } Any suggestions for how to best handle this dependency? We could say that JMSAdaptor by default uses AMQ, but it could be subclassed for other JMS providers. Or we could make JMSAdaptor abstact and have a concrete ActiveMQJMSAdaptor subclass.
        Hide
        eyang Eric Yang added a comment -

        For the initial implementation, I am fine with using ActiveMQ as default. When this adaptor is more matured, we can clean up and create JMSAdaptor as abstract class, and expose as something that third party can implement.

        TestJMSAdaptor failed on my machine. The error message is attached to this jira.

        Show
        eyang Eric Yang added a comment - For the initial implementation, I am fine with using ActiveMQ as default. When this adaptor is more matured, we can clean up and create JMSAdaptor as abstract class, and expose as something that third party can implement. TestJMSAdaptor failed on my machine. The error message is attached to this jira.
        Hide
        billgraham Bill Graham added a comment -

        Here's CHUKWA-469.2.patch.

        Show
        billgraham Bill Graham added a comment - Here's CHUKWA-469 .2.patch.
        Hide
        asrabkin Ari Rabkin added a comment -

        I just committed this to TRUNK. Thanks, Bill!

        Show
        asrabkin Ari Rabkin added a comment - I just committed this to TRUNK. Thanks, Bill!

          People

          • Assignee:
            billgraham Bill Graham
            Reporter:
            billgraham Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development