Uploaded image for project: 'Chukwa'
  1. Chukwa
  2. CHUKWA-715

Implement Oozie Adaptor to collect oozie instrumentation info

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.6.0
    • Component/s: Data Collection
    • Labels:
      None

      Description

      Oozie exposes metrics via rest urls. And hence this feature will allow to collect the instrumentation information to chukwa and monitor oozie.

      1. chukwa-715.patch.0
        24 kB
        Sreepathi Prasanna
      2. chukwa-715.patch.1
        26 kB
        Sreepathi Prasanna

        Activity

        Hide
        sree2k Sreepathi Prasanna added a comment -

        Have re-factored the code a bit and have added oozieadaptor and a corresponding test case.

        Show
        sree2k Sreepathi Prasanna added a comment - Have re-factored the code a bit and have added oozieadaptor and a corresponding test case.
        Hide
        eyang Eric Yang added a comment -

        TestRestAdaptor.java and OozieAdaptor.java are missing license file.

        ChukwaConfiguration class is designed to read ChukwaConfiguration files. It looks like ChukwaUtil does something similar but not exactly. I think it could be optimized to use ChukwaConfiguration class to reduce some duplications.

        Show
        eyang Eric Yang added a comment - TestRestAdaptor.java and OozieAdaptor.java are missing license file. ChukwaConfiguration class is designed to read ChukwaConfiguration files. It looks like ChukwaUtil does something similar but not exactly. I think it could be optimized to use ChukwaConfiguration class to reduce some duplications.
        Hide
        sree2k Sreepathi Prasanna added a comment -

        TestRestAdaptor.java and OozieAdaptor.java are missing license file.

        Have added the license files.

        ChukwaConfiguration class is designed to read ChukwaConfiguration files. It looks like ChukwaUtil does something similar but not exactly. I think it could be optimized to use ChukwaConfiguration class to reduce some duplications.

        the readConfig() method in ChukwaAgent.java might be reused in many places. To avoid code duplication, i have placed this method in ChukwaUtil class. Now both oozieAdaptor and chukwaAgent classes can read the chukwaconfiguration code using this readConfiguration method in chukwaUtil class. ( essentially, refactoring)

        Show
        sree2k Sreepathi Prasanna added a comment - TestRestAdaptor.java and OozieAdaptor.java are missing license file. Have added the license files. ChukwaConfiguration class is designed to read ChukwaConfiguration files. It looks like ChukwaUtil does something similar but not exactly. I think it could be optimized to use ChukwaConfiguration class to reduce some duplications. the readConfig() method in ChukwaAgent.java might be reused in many places. To avoid code duplication, i have placed this method in ChukwaUtil class. Now both oozieAdaptor and chukwaAgent classes can read the chukwaconfiguration code using this readConfiguration method in chukwaUtil class. ( essentially, refactoring)
        Hide
        eyang Eric Yang added a comment -

        +1

        Show
        eyang Eric Yang added a comment - +1
        Hide
        eyang Eric Yang added a comment -

        I just committed this, thanks Sreepathi.

        Show
        eyang Eric Yang added a comment - I just committed this, thanks Sreepathi.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Chukwa-trunk #506 (See https://builds.apache.org/job/Chukwa-trunk/506/)
        CHUKWA-715. Added Oozie Adaptor for collecting Oozie metrics. (Sreepathi Prasanna via Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612617)

        • /chukwa/trunk/CHANGES.txt
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/OozieAdaptor.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaAgent.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/util/ChukwaUtil.java
        • /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/util/RestUtil.java
        • /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/adaptor/TestOozieAdaptor.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Chukwa-trunk #506 (See https://builds.apache.org/job/Chukwa-trunk/506/ ) CHUKWA-715 . Added Oozie Adaptor for collecting Oozie metrics. (Sreepathi Prasanna via Eric Yang) (eyang: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1612617 ) /chukwa/trunk/CHANGES.txt /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/adaptor/OozieAdaptor.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/datacollection/agent/ChukwaAgent.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/util/ChukwaUtil.java /chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/util/RestUtil.java /chukwa/trunk/src/test/java/org/apache/hadoop/chukwa/datacollection/adaptor/TestOozieAdaptor.java

          People

          • Assignee:
            sree2k Sreepathi Prasanna
            Reporter:
            sree2k Sreepathi Prasanna
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development