Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Java, WS-DM Specification

      Description

      Create a WS-DM (Web Services-Distributed Management) interface that will receive management Data through the AMQP mgmt protocol and translate those into WS-DM.

      The main objective of introducing this AMQP-WSDM bridge in Apache Qpid is to allow any qpid broker to be hooked into any WS-DM console and allow for all the instrumentation data to be read, all the methods to be invoked, and the configuration to be updated through standard WS-DM console.

      This project will take the schema from AMQP-mgnt and then dynamically expose those over JMX and WS-DM. This means that any of the management data from the broker can be managed by any JMX console or WS-DM console.

      "Implementation of WSDM specification(s) (Management of Web Service & Management Using Web Service ) and integrate this in the Apache Qpid will be the major tasks in this project."

      1. qpid_JMX-WSDM.ppt
        538 kB
        Rahul Mehta
      2. Qpid-JMX.doc
        376 kB
        Rahul Mehta
      3. QpidJMX.zip
        47 kB
        Rahul Mehta
      4. README.txt
        6 kB
        Rahul Mehta
      5. Screenshot.png
        113 kB
        Rahul Mehta

        Activity

        Hide
        Ted Ross added a comment -

        Rahul,

        I am in the process of reviewing and cleaning up the documentation for the management of the C++ broker. One of the changes that will be committed in the next couple of days is a change to the schema of the management-schema document. This will be done to make the management schema easier to read and to provide some new functionality.

        I don't know if you plan to use the schema document directly (or learn the schema via the protocol) but wanted to give you a heads-up in case this change affects you.

        -Ted

        Show
        Ted Ross added a comment - Rahul, I am in the process of reviewing and cleaning up the documentation for the management of the C++ broker. One of the changes that will be committed in the next couple of days is a change to the schema of the management-schema document. This will be done to make the management schema easier to read and to provide some new functionality. I don't know if you plan to use the schema document directly (or learn the schema via the protocol) but wanted to give you a heads-up in case this change affects you. -Ted
        Hide
        Rahul Mehta added a comment -

        A small presentation (initial first draft) on this project, containing little detail about the project, technologies and a high-level architecture.
        This will be formulated with more pertinent information soon.

        Your comments are welcome.
        Thanks.

        /rahul

        Show
        Rahul Mehta added a comment - A small presentation (initial first draft) on this project, containing little detail about the project, technologies and a high-level architecture. This will be formulated with more pertinent information soon. Your comments are welcome. Thanks. /rahul
        Hide
        Rahul Mehta added a comment -

        Thanks Ted for intimating me about this.

        Currently, I am trying to retrieve schema from the protocol.

        Have you also uploaded the new schema on the SVN. (on this link: https://svn.apache.org/repos/asf/incubator/qpid/trunk/qpid/specs/management-schema.xml)

        So that I can just map what i get from the broker.

        /rahul

        Show
        Rahul Mehta added a comment - Thanks Ted for intimating me about this. Currently, I am trying to retrieve schema from the protocol. Have you also uploaded the new schema on the SVN. (on this link: https://svn.apache.org/repos/asf/incubator/qpid/trunk/qpid/specs/management-schema.xml ) So that I can just map what i get from the broker. /rahul
        Hide
        Rahul Mehta added a comment -

        This package attachment contains:

        1. Java Management Client code - that sends the management request messages to the C++ Qpid broker and receives the management response messages (more precisely schema for the package and classes known to broker)

        2. A README.txt file about how to download, install and run the required component and this management Client.

        Show
        Rahul Mehta added a comment - This package attachment contains: 1. Java Management Client code - that sends the management request messages to the C++ Qpid broker and receives the management response messages (more precisely schema for the package and classes known to broker) 2. A README.txt file about how to download, install and run the required component and this management Client.
        Hide
        Rahul Mehta added a comment -

        This package attachment contains:

        1. Java Management Client sorce code - that sends the management request messages to the C++ Qpid broker and receives the management response messages (more precisely schema for the package and classes known to broker)

        2. A README.txt file about how to download, install and run the required component and this management Client.

        Show
        Rahul Mehta added a comment - This package attachment contains: 1. Java Management Client sorce code - that sends the management request messages to the C++ Qpid broker and receives the management response messages (more precisely schema for the package and classes known to broker) 2. A README.txt file about how to download, install and run the required component and this management Client.
        Hide
        Rahul Mehta added a comment -

        An initial draft of documentation about the JMX-Bridge project is attached. This will be updated and reviewed periodically on the basis of ongoing work.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - An initial draft of documentation about the JMX-Bridge project is attached. This will be updated and reviewed periodically on the basis of ongoing work. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        An initial draft of documentation about the JMX-Bridge project is attached.
        This will be updated and reviewed periodically on the basis of ongoing work.

        I believe, your feedback will be helpful.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - An initial draft of documentation about the JMX-Bridge project is attached. This will be updated and reviewed periodically on the basis of ongoing work. I believe, your feedback will be helpful. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Updated with architecture and management client slides.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Updated with architecture and management client slides. Thanks. BR, Rahul
        Hide
        Arnaud Simon added a comment -

        Rahul,

        The code you have attached does not compile. Could you solve that?

        Show
        Arnaud Simon added a comment - Rahul, The code you have attached does not compile. Could you solve that?
        Hide
        Rahul Mehta added a comment - - edited

        Hi Arnaud,

        I checked the code, its running corrctly on my system. What error you are experiencing?

        In addition, I have attached the README file so you can follow the steps to run the "Management Client"

        Many Thanks.

        Rahul

        Show
        Rahul Mehta added a comment - - edited Hi Arnaud, I checked the code, its running corrctly on my system. What error you are experiencing? In addition, I have attached the README file so you can follow the steps to run the "Management Client" Many Thanks. Rahul
        Hide
        Arnaud Simon added a comment -

        Hi,

        I am looking at apache.zip and I am seeing the following errors:

        • ManagementDecoder cannot extend AbstractDecoder as it is not within the same package
        • same for ManagementEncoder
        • The above issues being solved, ManagementClient has a type mismatch issue at line 463

        I may not be looking at the right code though.

        One question on the code I am looking at:

        • why is the run method of ManagementClient infinitely looping creating destinations?

        Arnaud

        Show
        Arnaud Simon added a comment - Hi, I am looking at apache.zip and I am seeing the following errors: ManagementDecoder cannot extend AbstractDecoder as it is not within the same package same for ManagementEncoder The above issues being solved, ManagementClient has a type mismatch issue at line 463 I may not be looking at the right code though. One question on the code I am looking at: why is the run method of ManagementClient infinitely looping creating destinations? Arnaud
        Hide
        Rahul Mehta added a comment -

        Supporting APIs are attached in src.zip file to run the "ManagementClient"

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Supporting APIs are attached in src.zip file to run the "ManagementClient" BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Updated Management Client - Removed the run method as Arnaud suggested.

        Show
        Rahul Mehta added a comment - Updated Management Client - Removed the run method as Arnaud suggested.
        Hide
        Arnaud Simon added a comment -

        Hi Rahul,

        Why did you attach your repo? It is huge! What you should add is a patch, simply run: svn diff > mypatch.txt and attach the generated patch.
        May I ask you to delete the src.zip attachment please?

        Moreover, I believe you should not need to update the repo but rather use a proper package naming.

        Thanks

        Arnaud

        Show
        Arnaud Simon added a comment - Hi Rahul, Why did you attach your repo? It is huge! What you should add is a patch, simply run: svn diff > mypatch.txt and attach the generated patch. May I ask you to delete the src.zip attachment please? Moreover, I believe you should not need to update the repo but rather use a proper package naming. Thanks Arnaud
        Hide
        Rahul Mehta added a comment -

        This is the new version of the attached code.

        Main updates:

        1. QueryListener - to receive the query response in the "reply" queue
        2. MangaementListener - to receives the management Configuration and Instrumentation Content Messages periodically
        3. ExposeMBean - expose the "queue" class as DynamicMbean on JMX agent and manager can view the management information over JMX console.. and "purge" method removes the number of messages (as an argument) provided by the manager at runtime and these changes are reflecting inside the broker.

        Note: in the same way I can expose the other classes. I am hoping to expose all classes of broker on JMX by next week.

        thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - This is the new version of the attached code. Main updates: 1. QueryListener - to receive the query response in the "reply" queue 2. MangaementListener - to receives the management Configuration and Instrumentation Content Messages periodically 3. ExposeMBean - expose the "queue" class as DynamicMbean on JMX agent and manager can view the management information over JMX console.. and "purge" method removes the number of messages (as an argument) provided by the manager at runtime and these changes are reflecting inside the broker. Note: in the same way I can expose the other classes. I am hoping to expose all classes of broker on JMX by next week. thanks. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Hello All,

        I uploaded the code on the JIRA - 955.

        There are three main files:

        1. QueryListener - which creates the three queues (management, reply, testQueue) and prompts user to quey (e.g. package, schema, method etc) and put the schema into the property file
        2. ManagementListener - this listens the Configuration and Instrumentation Content Messages for all the class that the broker knows. In the present situation, it gets all the attributes for the queue (testQueue) and write into the property file so that we can expose it (by using ExposeMBean.java)
        3. ExposeMBean - is used to get the elements from the property file and expose them on the JMX agent by Running (ExposeMBeanMain.java)

        You can open the jconsole to view all the elements of the queue class "testQueue", you can also invoke the "purge" method by supplying an argument "request" (number of messages you want to delete). This change will directly reflect into the broker.

        Please note: you cannot modify the elements value via console as this setAttribute() mapping is not implemented in the broker. (as Ted confirmed)

        One last thing, if you run these program then please change the objectID in the QueryListener, ManagementListener and ExposeQueue for the testQueue queue (as it is hardcoded) but in the next version it would be dynamic. (with all the classes)

        There is one more program, which I reuse from already existing api is DirectProducer for testing which is used to put few messages in the testQueue

        How to run this:

        1. Run the QueryListener.java if you want to send any management request and receive the management response on the "reply" queue. This will create the queues automatically by calling Queue.java

        2. Run the ManagementListener if you want to receive the sporadic Configuration and Instrumentation Content Messages for all the class that the broker knows. It will also create the queues.

        3. You can run the qpid-tool to see the objectId of the testQueue and change in the all three files (QueryListener.java, ManagementListener and ExposeMBean.java) as it is hard coded for the time being.  . but in the next version will be fixed.

        4. Run the DirectProducer.java to put some messages in the testQueue

        5. step -2

        6. see in the directory structure ./conf/exposeProperty.prop file, you will receive the all queue elements in it.

        7. run the ExposeMBeanMain.java to expose all the elements of the property file on JMX agent

        8. open jconsole and view all elements of the queue, you can also invoke the purge method to purge the messaged from the testQueue

        Next Step- by next week I am hoping to expose all the classes dynamically and fix issues.

        Thanks to Ted for guiding me and notifying the changes in management protocol otherwise I was caught with the old version of broker and apis.

        You feedback is required.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello All, I uploaded the code on the JIRA - 955. There are three main files: 1. QueryListener - which creates the three queues (management, reply, testQueue) and prompts user to quey (e.g. package, schema, method etc) and put the schema into the property file 2. ManagementListener - this listens the Configuration and Instrumentation Content Messages for all the class that the broker knows. In the present situation, it gets all the attributes for the queue (testQueue) and write into the property file so that we can expose it (by using ExposeMBean.java) 3. ExposeMBean - is used to get the elements from the property file and expose them on the JMX agent by Running (ExposeMBeanMain.java) You can open the jconsole to view all the elements of the queue class "testQueue", you can also invoke the "purge" method by supplying an argument "request" (number of messages you want to delete). This change will directly reflect into the broker. Please note: you cannot modify the elements value via console as this setAttribute() mapping is not implemented in the broker. (as Ted confirmed) One last thing, if you run these program then please change the objectID in the QueryListener, ManagementListener and ExposeQueue for the testQueue queue (as it is hardcoded) but in the next version it would be dynamic. (with all the classes) There is one more program, which I reuse from already existing api is DirectProducer for testing which is used to put few messages in the testQueue How to run this: 1. Run the QueryListener.java if you want to send any management request and receive the management response on the "reply" queue. This will create the queues automatically by calling Queue.java 2. Run the ManagementListener if you want to receive the sporadic Configuration and Instrumentation Content Messages for all the class that the broker knows. It will also create the queues. 3. You can run the qpid-tool to see the objectId of the testQueue and change in the all three files (QueryListener.java, ManagementListener and ExposeMBean.java) as it is hard coded for the time being.  . but in the next version will be fixed. 4. Run the DirectProducer.java to put some messages in the testQueue 5. step -2 6. see in the directory structure ./conf/exposeProperty.prop file, you will receive the all queue elements in it. 7. run the ExposeMBeanMain.java to expose all the elements of the property file on JMX agent 8. open jconsole and view all elements of the queue, you can also invoke the purge method to purge the messaged from the testQueue Next Step- by next week I am hoping to expose all the classes dynamically and fix issues. Thanks to Ted for guiding me and notifying the changes in management protocol otherwise I was caught with the old version of broker and apis. You feedback is required. Thanks. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Hello All,

        This version of the project successfully exposes all the "Configuration and Instrumentation Content Messages" and "methods" of all the objects of C++ QPID BROKER on the JMX agent. To view them please run jconsole in parallel. Invocation of methods replicate the changes inside the broker. To view them please run the "qpid-tool" management tool as at this moment notification is not implemented in this project. This was the objective of my GSoC project : "fetching all the management information of all the objects and expose them on the JMX".

        I am also attaching an output image of this project. I will also attach the README.txt file tomorrow so that you can run the project and see the flow.

        I will also update the documentation and presentation which are already attached to this JIRA.

        Please feel free to leave your feedback.

        Thank you.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello All, This version of the project successfully exposes all the "Configuration and Instrumentation Content Messages" and "methods" of all the objects of C++ QPID BROKER on the JMX agent. To view them please run jconsole in parallel. Invocation of methods replicate the changes inside the broker. To view them please run the "qpid-tool" management tool as at this moment notification is not implemented in this project. This was the objective of my GSoC project : "fetching all the management information of all the objects and expose them on the JMX". I am also attaching an output image of this project. I will also attach the README.txt file tomorrow so that you can run the project and see the flow. I will also update the documentation and presentation which are already attached to this JIRA. Please feel free to leave your feedback. Thank you. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Attached is the screen shot of the project output.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Attached is the screen shot of the project output. Thanks. BR, Rahul
        Hide
        Carl Trieloff added a comment -

        Looks like it is coming together, thx for png

        Carl.

        Show
        Carl Trieloff added a comment - Looks like it is coming together, thx for png Carl.
        Hide
        Rahul Mehta added a comment -

        Hello All,

        README.txt file is attached so that everyone can read it and run this project.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello All, README.txt file is attached so that everyone can read it and run this project. Thanks. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Thank you Carl for your comment.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Thank you Carl for your comment. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Hello All,

        Presentation (ppt) on this project is updated for your high-level understanding.

        Thank you.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello All, Presentation (ppt) on this project is updated for your high-level understanding. Thank you. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Hello All,

        Documentation is updated for your further understanding on this project.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello All, Documentation is updated for your further understanding on this project. Thanks. BR, Rahul
        Hide
        Rahul Mehta added a comment -

        Hello,

        At this time we are exposing only one instance of each class.. so it means we are exposing 11 objects sent by broker. there are no multiple instance of the classes (i.e. just "management" queue instance for "queue" object, not the "reply queue" or any other queues known to broker).

        So now I am implementing the factory design pattern in this project to make it more dynamic at runtime. The objective will be whatever the object we will receive from the broker, this application will expose on the JMX agent.

        Note: this is not in the scope of GSoC... its just improvement.

        Thanks.

        BR,
        Rahul

        Show
        Rahul Mehta added a comment - Hello, At this time we are exposing only one instance of each class.. so it means we are exposing 11 objects sent by broker. there are no multiple instance of the classes (i.e. just "management" queue instance for "queue" object, not the "reply queue" or any other queues known to broker). So now I am implementing the factory design pattern in this project to make it more dynamic at runtime. The objective will be whatever the object we will receive from the broker, this application will expose on the JMX agent. Note: this is not in the scope of GSoC... its just improvement. Thanks. BR, Rahul
        Hide
        Robbie Gemmell added a comment -

        Closing out old issue.

        Show
        Robbie Gemmell added a comment - Closing out old issue.

          People

          • Assignee:
            Unassigned
            Reporter:
            Rahul Mehta
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development