Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1017

Optimization: Reduce Overhead from ReflectionUtils.newInstance

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.12.0
    • Component/s: util
    • Labels:
      None

      Description

      I found that a significant amount of time on my project was being spent in creating constructors for each row of data. I dramatically optimized this performance by creating a simple WeakHashMap to cache constructors by class. For example, in a sample job I find that ReflectionUtils.newInstance takes 200 ms (2% of total) with the cache enabled, but it uses 900 ms (6% of total) without the cache.

        Attachments

        1. cacheCtor.patch
          5 kB
          Ron Bodkin
        2. cacheCtor.patch
          5 kB
          Ron Bodkin
        3. TestReflectionUtils.java
          2 kB
          Ron Bodkin
        4. ReflectionUtils.patch.txt
          2 kB
          Ron Bodkin

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              rbodkin Ron Bodkin
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: