Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1881

Improve TaskTrackerInstrumentation

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • None
    • None
    • None

    Description

      The TaskTrackerInstrumentation class provides a useful way to capture key events at the TaskTracker for use in various reporting tools, but it is currently rather limited, because only one TaskTrackerInstrumentation can be added to a given TaskTracker and this objects receives minimal information about tasks (only their IDs). I propose enhancing the functionality through two changes:

      1. Support a comma-separated list of TaskTrackerInstrumentation classes rather than just a single one in the JobConf, and report events to all of them.
      2. Make the reportTaskLaunch and reportTaskEnd methods in TaskTrackerInstrumentation receive a reference to a whole Task object rather than just its TaskAttemptID. It might also be useful to make the latter receive the task's final state, i.e. failed, killed, or successful.

      I'm just posting this here to get a sense of whether this is a good idea. If people think it's okay, I will make a patch against trunk that implements these changes.

      Attachments

        1. mapreduce-1881-v4.patch
          20 kB
          Matei Alexandru Zaharia
        2. mapreduce-1881-v3.patch
          12 kB
          Matei Alexandru Zaharia
        3. mapreduce-1881-v2b.patch
          7 kB
          Matei Alexandru Zaharia
        4. mapreduce-1881-v2.patch
          7 kB
          Matei Alexandru Zaharia
        5. mapreduce-1881.patch
          7 kB
          Matei Alexandru Zaharia

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            matei Matei Alexandru Zaharia
            matei Matei Alexandru Zaharia
            Votes:
            0 Vote for this issue
            Watchers:
            19 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment