Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2855

ResourceBundle lookup during counter name resolution takes a lot of time

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.2
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time.

      Description

      Loading a job status page in trunk takes a lot of time, and it seems like most of the time is spent resolving counter names. Looking through the JDK source, ResourceBundle.getBundle(String) ends up calling getClassContext() which is not very efficient. I think if we pass our own classloader manually it will be faster. In Counters.incrAllCounters, we may also be able to avoid setting the counter name if one is already set.

      1. MR2855.txt
        1 kB
        Siddharth Seth

        Issue Links

          Activity

          Todd Lipcon created issue -
          Todd Lipcon made changes -
          Field Original Value New Value
          Assignee Todd Lipcon [ tlipcon ]
          Siddharth Seth made changes -
          Attachment MR2855.txt [ 12515689 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.23.2 [ 12319851 ]
          Labels mrv2
          Vinod Kumar Vavilapalli made changes -
          Assignee Siddharth Seth [ sseth ]
          Vinod Kumar Vavilapalli made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Release Note Passing a cached class-loader to ResourceBundle creator to minimize counter names lookup time.
          Fix Version/s 0.23.2 [ 12319851 ]
          Resolution Fixed [ 1 ]
          Karthik Kambatla made changes -
          Link This issue is cloned as MAPREDUCE-4565 [ MAPREDUCE-4565 ]

            People

            • Assignee:
              Siddharth Seth
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development