Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2932

Missing instrumentation plugin class shouldn't crash the TT startup per design

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Trivial Trivial
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.1.0
    • Component/s: tasktracker
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Per the implementation of the TaskTracker instrumentation plugin implementation (from 2008), a ClassNotFoundException during loading up of an configured TaskTracker instrumentation class shouldn't have hampered TT start up at all.

      But, there is one class-fetching call outside try/catch, which makes TT fall down with a RuntimeException if there's a class not found. Would be good to include this line into the try/catch itself.

      Strace would appear as:

      2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not start task tracker because java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.mapred.CustomInstPlugin 
      at org.apache.hadoop.conf.Configuration.getClass(Configuration.java) 
      at org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java) 
      at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java) 
      
      1. 0.20-MAPREDUCE-2932.r3.diff
        5 kB
        Harsh J
      2. 0.20-MAPREDUCE-2932.r2.diff
        5 kB
        Harsh J
      3. 0.20-MAPREDUCE-2932.r1.diff
        2 kB
        Harsh J

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        10m 26s 1 Harsh J 05/Sep/11 17:16
        Patch Available Patch Available Resolved Resolved
        17d 3h 7m 1 Eli Collins 22/Sep/11 20:23
        Resolved Resolved Closed Closed
        390d 23h 3m 1 Matt Foley 17/Oct/12 19:27
        Matt Foley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop-1.1.0.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop-1.1.0.
        Eli Collins made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags [Reviewed]
        Resolution Fixed [ 1 ]
        Hide
        Eli Collins added a comment -

        I've committed this. Thanks Harsh!

        Show
        Eli Collins added a comment - I've committed this. Thanks Harsh!
        Hide
        Eli Collins added a comment -

        +1 looks good. I ran the tests locally.

        Show
        Eli Collins added a comment - +1 looks good. I ran the tests locally.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12496060/0.20-MAPREDUCE-2932.r3.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/819//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12496060/0.20-MAPREDUCE-2932.r3.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/819//console This message is automatically generated.
        Harsh J made changes -
        Attachment 0.20-MAPREDUCE-2932.r3.diff [ 12496060 ]
        Hide
        Harsh J added a comment -

        Made tests j4-style. Will keep this in mind next time and not do just a copy-paste-edit job

        Show
        Harsh J added a comment - Made tests j4-style. Will keep this in mind next time and not do just a copy-paste-edit job
        Hide
        Eli Collins added a comment -

        Mind making your new test junit4 sytle? Otherwise, looks great!

        Show
        Eli Collins added a comment - Mind making your new test junit4 sytle? Otherwise, looks great!
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12495258/0.20-MAPREDUCE-2932.r2.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/803//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12495258/0.20-MAPREDUCE-2932.r2.diff against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/803//console This message is automatically generated.
        Harsh J made changes -
        Attachment 0.20-MAPREDUCE-2932.r2.diff [ 12495258 ]
        Hide
        Harsh J added a comment -

        Eli,

        Two tests added. Also noticed there was no existing testcase for TT instrumentation plugins at all?

        Show
        Harsh J added a comment - Eli, Two tests added. Also noticed there was no existing testcase for TT instrumentation plugins at all?
        Eli Collins made changes -
        Fix Version/s 0.20.206.0 [ 12317960 ]
        Affects Version/s 0.20.2 [ 12314205 ]
        Hide
        Eli Collins added a comment -

        Patch looks good. Can you add a test case similar to TestJobTrackerInstrumentation?

        Show
        Eli Collins added a comment - Patch looks good. Can you add a test case similar to TestJobTrackerInstrumentation?
        Hide
        Harsh J added a comment -

        I s'pose NodeManagerMetrics is the newer equivalent of the older, pluggable instrumentation of TT. Since NodeManagerMetrics is not an interface, and nor does the framework allow things to just be plugged in (to merely override the metrics class) for this, I do not think this problem exists MR2 (0.23/trunk).

        Show
        Harsh J added a comment - I s'pose NodeManagerMetrics is the newer equivalent of the older, pluggable instrumentation of TT. Since NodeManagerMetrics is not an interface, and nor does the framework allow things to just be plugged in (to merely override the metrics class) for this, I do not think this problem exists MR2 (0.23/trunk).
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12493041/0.20-MAPREDUCE-2932.r1.diff
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/607//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493041/0.20-MAPREDUCE-2932.r1.diff against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/607//console This message is automatically generated.
        Harsh J made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Harsh J made changes -
        Field Original Value New Value
        Attachment 0.20-MAPREDUCE-2932.r1.diff [ 12493041 ]
        Hide
        Harsh J added a comment -

        Patch for 0.20-security. Unsure if required for 0.22 and for any of new components of 0.23+, will have to investigate.

        Show
        Harsh J added a comment - Patch for 0.20-security. Unsure if required for 0.22 and for any of new components of 0.23+, will have to investigate.
        Harsh J created issue -

          People

          • Assignee:
            Harsh J
            Reporter:
            Harsh J
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development