Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      In the context of Hadoop-On-Demand (HOD), we want to be able to aggregate metrics across the machines in the virtual cluster that corresponds to a HOD session. To enable this, we'd like to tag all the map/reduce related metrics with a session id. The session id would be a string created by HOD (it might be a Torque job id) and put into the hadoop-site.xml file for the virtual cluster.

      The default value for the session id would be the empty string, so that non-HOD usage of the metrics is not affected accept for the addition of a tag which is always the empty string.

      1. 1518.patch
        5 kB
        David Bowen
      2. patch.txt
        5 kB
        David Bowen

        Activity

        Hide
        David Bowen added a comment -

        This patch adds the session id to all the map/reduce related metrics.

        Show
        David Bowen added a comment - This patch adds the session id to all the map/reduce related metrics.
        Show
        Hadoop QA added a comment - +1 http://issues.apache.org/jira/secure/attachment/12360319/patch.txt applied and successfully tested against trunk revision r549577. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/319/testReport/ Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/319/console
        Hide
        Doug Cutting added a comment -

        The javadoc for set/getSessionId should describe what the session id is used for, as should the description in hadoop-default.xml. Will this actually be set in site configurations, or only programatically? If it's only set programatically, then it doesn't need to be in hadoop-default.xml, which primarily exists to document available parameters for site configuration, not necessarily every parameter ever used by programs.

        Show
        Doug Cutting added a comment - The javadoc for set/getSessionId should describe what the session id is used for, as should the description in hadoop-default.xml. Will this actually be set in site configurations, or only programatically? If it's only set programatically, then it doesn't need to be in hadoop-default.xml, which primarily exists to document available parameters for site configuration, not necessarily every parameter ever used by programs.
        Hide
        David Bowen added a comment -

        Improved patch. Removed the change to hadoop-default.xml, and added comments to JobConf.getSessionId to explain the intended purpose of the session identifier.

        Show
        David Bowen added a comment - Improved patch. Removed the change to hadoop-default.xml, and added comments to JobConf.getSessionId to explain the intended purpose of the session identifier.
        Show
        Hadoop QA added a comment - +1 http://issues.apache.org/jira/secure/attachment/12360329/1518.patch applied and successfully tested against trunk revision r549624. Test results: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/321/testReport/ Console output: http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/321/console
        Hide
        Doug Cutting added a comment -

        I just committed this. Thanks, David!

        Show
        Doug Cutting added a comment - I just committed this. Thanks, David!
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-Nightly #133 (See http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Nightly/133/ )

          People

          • Assignee:
            David Bowen
            Reporter:
            David Bowen
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development