Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-1017

Optimization: Reduce Overhead from ReflectionUtils.newInstance

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.12.0
    • util
    • None

    Description

      I found that a significant amount of time on my project was being spent in creating constructors for each row of data. I dramatically optimized this performance by creating a simple WeakHashMap to cache constructors by class. For example, in a sample job I find that ReflectionUtils.newInstance takes 200 ms (2% of total) with the cache enabled, but it uses 900 ms (6% of total) without the cache.

      Attachments

        1. cacheCtor.patch
          5 kB
          Ron Bodkin
        2. cacheCtor.patch
          5 kB
          Ron Bodkin
        3. TestReflectionUtils.java
          2 kB
          Ron Bodkin
        4. ReflectionUtils.patch.txt
          2 kB
          Ron Bodkin

        Activity

          People

            Unassigned Unassigned
            rbodkin Ron Bodkin
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: