Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5696

Add Localization counters to MR

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • mrv2
    • None

    Description

      Users are often unaware of localization cost that their jobs incur. To measure effectiveness of localization caches it is necessary to expose the overhead in the form of user-visible metrics. The purpose of this JIRA is to compliment YARN-1529. While YARN-1529 attempts to provide a cluster-wide view to cluster admins, this JIRA focuses on exposing the localization overhead on per-job basis to the job owner/user.

      Attachments

        1. MAPREDUCE-5696.v02.patch
          12 kB
          Gera Shegalov
        2. MAPREDUCE-5696.v01.patch
          20 kB
          Gera Shegalov
        3. LocalizationCounters.png
          139 kB
          Gera Shegalov

        Issue Links

          requires

          Improvement - An improvement or enhancement to an existing feature or task. YARN-1529 Add Localization overhead metrics to NM

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              jira.shegalov Gera Shegalov
              jira.shegalov Gera Shegalov
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated: