Chukwa
  1. Chukwa
  2. CHUKWA-520

Metrics graphing REST api for HBase

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.5.0
    • Component/s: User Interface
    • Labels:
      None
    • Environment:

      Java 6, Mac OS X 10.6

    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      HICC chart.jsp has the ability to graph metrics from web service API. The usage pattern looks like this:

      /hicc/jsp/chart.jsp?data=[url to json data]
      

      The expected data format for the data url looks like this:

      { "name": "series name", "data": [[x,y],[x2,y2],...[x1000,y1000]]}
      

      Metrics graphing REST api should load data from HBase and convert to the above JSON output format for graphing.
      The suggested REST API looks like this:

      /hicc/v1/metrics/[table_name]/[column_family]:[column]/[row_key]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
      

      row_key can be supplied like a macro, i.e. [host]. This will pick up selected host from HICC session variable.

      1. chukwa_hello_world.png
        217 kB
        Eric Yang
      2. CHUKWA-520-1.patch
        33 kB
        Eric Yang
      3. chart_example.png
        127 kB
        Eric Yang
      4. CHUKWA-520.patch
        8 kB
        Eric Yang

        Activity

        Hide
        Eric Yang added a comment -

        Proposed REST API for metrics.

        Show
        Eric Yang added a comment - Proposed REST API for metrics.
        Hide
        Eric Yang added a comment -

        Screenshot of the rest api in action.

        Show
        Eric Yang added a comment - Screenshot of the rest api in action.
        Hide
        Ari Rabkin added a comment -

        This sounds useful. Can you also add some user documentation for this somewhere? (Perhaps in the admin guide?)

        Show
        Ari Rabkin added a comment - This sounds useful. Can you also add some user documentation for this somewhere? (Perhaps in the admin guide?)
        Hide
        Eric Yang added a comment -

        Sure, most of the document will go into javadoc, but I also would like to start developer document for hicc. I think I will start HICC document in wiki then move to xml document when API is ready for public.

        Show
        Eric Yang added a comment - Sure, most of the document will go into javadoc, but I also would like to start developer document for hicc. I think I will start HICC document in wiki then move to xml document when API is ready for public.
        Hide
        Ari Rabkin added a comment -

        Hm. Javadoc doesn't really strike me as a good way to provide documentation for users. Do we need a separate HICC user guide?

        Show
        Ari Rabkin added a comment - Hm. Javadoc doesn't really strike me as a good way to provide documentation for users. Do we need a separate HICC user guide?
        Hide
        Eric Yang added a comment -

        Yes, we need a separate HICC user guide CHUKWA-520 to address this issue.

        Show
        Eric Yang added a comment - Yes, we need a separate HICC user guide CHUKWA-520 to address this issue.
        Hide
        Eric Yang added a comment -

        The finalized design:

        1. Get series data by row key.

        /hicc/v1/metrics/series/[table]/[column]/rowkey/[rkey]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
        

        2. Get series data by HICC session macro.

        /hicc/v1/metrics/series/[table]/[column]/session/[sessionKey]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
        

        3. Get list of HBase tables

        /hicc/v1/metrics/schema
        

        4. Get list of column family

        /hicc/v1/metrics/schema/[table]
        

        5. Get list of columns in a column family

        /hicc/v1/metrics/schema/[table]/[family]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
        

        6. Get list of row key associated with a column

        /hicc/v1/metrics/schema/[table]/[column]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
        
        Show
        Eric Yang added a comment - The finalized design: 1. Get series data by row key. /hicc/v1/metrics/series/[table]/[column]/rowkey/[rkey]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss] 2. Get series data by HICC session macro. /hicc/v1/metrics/series/[table]/[column]/session/[sessionKey]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss] 3. Get list of HBase tables /hicc/v1/metrics/schema 4. Get list of column family /hicc/v1/metrics/schema/[table] 5. Get list of columns in a column family /hicc/v1/metrics/schema/[table]/[family]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss] 6. Get list of row key associated with a column /hicc/v1/metrics/schema/[table]/[column]?start=[yyyyMMddHHmmss]&end=[yyyyMMddHHmmss]
        Hide
        Eric Yang added a comment -
        • Refined data schema discovery tool.
        • Include graph explorer for exploring data in HBase.
        Show
        Eric Yang added a comment - Refined data schema discovery tool. Include graph explorer for exploring data in HBase.
        Hide
        Eric Yang added a comment -

        graph_explorer in action.

        Show
        Eric Yang added a comment - graph_explorer in action.
        Hide
        Ari Rabkin added a comment -

        Neat. Let's commit it. +1

        Show
        Ari Rabkin added a comment - Neat. Let's commit it. +1
        Hide
        Jerome Boulon added a comment -

        Nice!
        Do you have some number at scale?

        Show
        Jerome Boulon added a comment - Nice! Do you have some number at scale?
        Hide
        Eric Yang added a comment -

        Jerome, could you be more specific? I don't understand the question.

        Show
        Eric Yang added a comment - Jerome, could you be more specific? I don't understand the question.
        Hide
        Eric Yang added a comment -

        I just committed this, thanks Ari.

        Show
        Eric Yang added a comment - I just committed this, thanks Ari.

          People

          • Assignee:
            Eric Yang
            Reporter:
            Eric Yang
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development