Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2629

Class loading quirk prevents inner class method compilation

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: task
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While profiling jobs like terasort and gridmix, I noticed that a
      method "org.apache.hadoop.mapreduce.task.ReduceContextImpl.access
      $000" is near the top. It turns out that this is because the
      ReduceContextImpl class has a member backupStore which is accessed
      from an inner class ReduceContextImpl$ValueIterator. Due to the way
      synthetic accessor methods work, every access of backupStore results
      in a call to access$000 to the outer class. For some portion of the
      run, backupStore is null and the BackupStore class has never been
      loaded by the reducer.

      Due to the way the Hotspot JVM inliner works, by default it will not
      inline a short method where the class of of the return value object
      is unloaded - if you use a debug JVM with -XX:+PrintCompilation you
      will see a failure reason message like "unloaded signature classes."
      This causes every call to ReduceContextImpl.access$000 to be executed
      in the interpreter for the handful of bytecodes to return the null
      backupStore.

        Attachments

        1. mr-2629.txt
          2 kB
          Todd Lipcon
        2. MAPREDUCE-2629.patch
          1 kB
          Eric Caspole

          Activity

            People

            • Assignee:
              ecaspole Eric Caspole
              Reporter:
              ecaspole Eric Caspole
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 24h
                24h
                Remaining:
                Remaining Estimate - 24h
                24h
                Logged:
                Time Spent - Not Specified
                Not Specified