Details

    • Type: Sub-task Sub-task
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: monitor
    • Labels:

      Description

      The monitor is awesome. It should die.

      I'm going to move other monitor tickets under this one (if I can), and create some requirement tickets.

      We would be better off putting our weight behind an existing monitoring program which can scale, if one exists.

      Hopefully we can combine tracing efforts and have a nicer distributed trace-based tool, too.

      For display functionality, lots of possibilities: Graphite, Cubism.js, D3.js (really, any number of really slick Javascript graphing libraries). For log collection, any number of distributed log management services out there too can serve as inspiration for functionality: statsd, logstash, cacti/rrdtool.

      Currently all of Accumulo monitoring information is exposed via JMX; a nice balance could be found leveraging the existing monitoring capabilities with JMXTrans (or equivalent) and applying a new GUI.

      Familiarity with Java and JMX would be ideal.

        Issue Links

          Activity

          Hide
          Andres Danter added a comment -

          Hi Eric,

          I'm hoping to get involved in the Summer of Code event and I would like to help enhance Accumulo. I noticed that several of the issues flagged as SoC ideas mostly revolve around shortcomings of Accumulo's monitor application. I think that the introduction of a new monitor to Accumulo is the way to go, and it would be a perfect project for SoC. I do not have an existing monitoring framework in mind for this, but that research on my part can be performed before I submit my proposal.

          Let me know what I need to do to start discussing my ideas and possible directions for this project with the Accumulo community.

          Thank you,

          Andres

          Show
          Andres Danter added a comment - Hi Eric, I'm hoping to get involved in the Summer of Code event and I would like to help enhance Accumulo. I noticed that several of the issues flagged as SoC ideas mostly revolve around shortcomings of Accumulo's monitor application. I think that the introduction of a new monitor to Accumulo is the way to go, and it would be a perfect project for SoC. I do not have an existing monitoring framework in mind for this, but that research on my part can be performed before I submit my proposal. Let me know what I need to do to start discussing my ideas and possible directions for this project with the Accumulo community. Thank you, Andres
          Hide
          John Vines added a comment -

          This is a lofty goal we continuously have. But there are bigger fish so it keeps getting pushed (BTW, pushing it to 1.7). I'm wondering if we should delist a fix version for it.

          Show
          John Vines added a comment - This is a lofty goal we continuously have. But there are bigger fish so it keeps getting pushed (BTW, pushing it to 1.7). I'm wondering if we should delist a fix version for it.
          Hide
          Josh Elser added a comment -

          The more I've been thinking about this, the more I'm leaning towards trying to migrate the "monitor" into some REST service and rehash the GUI to use that service instead of making direct calls to Accumulo. This would provide an interface to monitoring consumers that isn't reliant on JMX (or some process that can convert the JMX for you like jmxtrans). It would also make it infinitely easier to write tests to ensure that we're returning correct metrics. Something that packages into a WAR would also be nice as it could be deployed into an existing application server (not sure if we had a ticket specifically for that elsewhere).

          Lots of alternatives for writing a webservice too: dropwizard, jersey, jax-rs initially come to mind.

          Show
          Josh Elser added a comment - The more I've been thinking about this, the more I'm leaning towards trying to migrate the "monitor" into some REST service and rehash the GUI to use that service instead of making direct calls to Accumulo. This would provide an interface to monitoring consumers that isn't reliant on JMX (or some process that can convert the JMX for you like jmxtrans). It would also make it infinitely easier to write tests to ensure that we're returning correct metrics. Something that packages into a WAR would also be nice as it could be deployed into an existing application server (not sure if we had a ticket specifically for that elsewhere). Lots of alternatives for writing a webservice too: dropwizard, jersey, jax-rs initially come to mind.
          Hide
          David Medinets added a comment -

          Is there any way to quantify the number of hours this kind of change might take? If the change were broken into phases, what might those phases look like?

          Show
          David Medinets added a comment - Is there any way to quantify the number of hours this kind of change might take? If the change were broken into phases, what might those phases look like?
          Hide
          Josh Elser added a comment -

          re: number of hours, it's obviously going to be relative to the amount of experience with the tool being used. I could see the work (that I proposed) being broken into the following:

          1. Identify metrics/API that we want to create from existing monitor (potentially what extra we also want to add)
          2. Stub out interfaces using library of choice.
          3. Begin writing implementations + tests for each interface

          I also forgot to mention that, with a good REST API, we could implement some version notion to provide stability as to the API we expose.

          Show
          Josh Elser added a comment - re: number of hours, it's obviously going to be relative to the amount of experience with the tool being used. I could see the work (that I proposed) being broken into the following: 1. Identify metrics/API that we want to create from existing monitor (potentially what extra we also want to add) 2. Stub out interfaces using library of choice. 3. Begin writing implementations + tests for each interface I also forgot to mention that, with a good REST API, we could implement some version notion to provide stability as to the API we expose.

            People

            • Assignee:
              Unassigned
              Reporter:
              Eric Newton
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:

                Development