Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2472

Extra whitespace in mapred.child.java.opts breaks JVM initialization

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2, 0.22.0
    • Fix Version/s: 0.22.0
    • Component/s: task-controller
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When creating taskjvm.sh, we split mapred.child.java.opts on " " and then create a quoted argument for each of those results. So, if you have an extra space anywhere in this configuration, you get an argument '' in the child command line, which the JVM interprets as an empty class name. This results in a ClassNotFoundException and the task cannot run.

      1. mapreduce-2472.1.patch
        4 kB
        Aaron T. Myers
      2. mapreduce-2472.0.patch
        4 kB
        Aaron T. Myers

        Activity

        Hide
        Aaron T. Myers added a comment -

        Patch which changes the split from being on spaces to any amount of whitespace. Add a few tests for this as well.

        Show
        Aaron T. Myers added a comment - Patch which changes the split from being on spaces to any amount of whitespace. Add a few tests for this as well.
        Hide
        Todd Lipcon added a comment -

        Hm, this seems to take care of extra intrnal whitespace, but I think we need a trim() in there too for trailing/leading whitespace

        Show
        Todd Lipcon added a comment - Hm, this seems to take care of extra intrnal whitespace, but I think we need a trim() in there too for trailing/leading whitespace
        Hide
        Aaron T. Myers added a comment -

        Thanks for the comments, Todd.

        Updated patch to address Todd's comments. Also refactored the tests a little bit and added tests to check for proper handling of leading or trailing whitespace.

        Show
        Aaron T. Myers added a comment - Thanks for the comments, Todd. Updated patch to address Todd's comments. Also refactored the tests a little bit and added tests to check for proper handling of leading or trailing whitespace.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12478184/mapreduce-2472.1.patch
        against trunk revision 1098853.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 2 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12478184/mapreduce-2472.1.patch against trunk revision 1098853. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 2 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/210//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        +1

        Show
        Todd Lipcon added a comment - +1
        Hide
        Todd Lipcon added a comment -

        Committed to trunk and branch 22

        Show
        Todd Lipcon added a comment - Committed to trunk and branch 22
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #656 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/656/)
        MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #656 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/656/ ) MAPREDUCE-2472 . Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-22-branch #45 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/45/)
        MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-22-branch #45 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-22-branch/45/ ) MAPREDUCE-2472 . Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #670 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/670/)
        MAPREDUCE-2472. Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #670 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/670/ ) MAPREDUCE-2472 . Extra whitespace in mapred.child.java.opts breaks JVM initialization. Contributed by Aaron T. Myers.

          People

          • Assignee:
            Aaron T. Myers
            Reporter:
            Todd Lipcon
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development