Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2629

Class loading quirk prevents inner class method compilation

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0, 0.22.0
    • Fix Version/s: 0.23.0
    • Component/s: task
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      While profiling jobs like terasort and gridmix, I noticed that a
      method "org.apache.hadoop.mapreduce.task.ReduceContextImpl.access
      $000" is near the top. It turns out that this is because the
      ReduceContextImpl class has a member backupStore which is accessed
      from an inner class ReduceContextImpl$ValueIterator. Due to the way
      synthetic accessor methods work, every access of backupStore results
      in a call to access$000 to the outer class. For some portion of the
      run, backupStore is null and the BackupStore class has never been
      loaded by the reducer.

      Due to the way the Hotspot JVM inliner works, by default it will not
      inline a short method where the class of of the return value object
      is unloaded - if you use a debug JVM with -XX:+PrintCompilation you
      will see a failure reason message like "unloaded signature classes."
      This causes every call to ReduceContextImpl.access$000 to be executed
      in the interpreter for the handful of bytecodes to return the null
      backupStore.

      1. MAPREDUCE-2629.patch
        1 kB
        Eric Caspole
      2. mr-2629.txt
        2 kB
        Todd Lipcon

        Activity

          People

          • Assignee:
            Eric Caspole
            Reporter:
            Eric Caspole
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Time Tracking

              Estimated:
              Original Estimate - 24h
              24h
              Remaining:
              Remaining Estimate - 24h
              24h
              Logged:
              Time Spent - Not Specified
              Not Specified

                Development