Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5253

Whitespace value entry in mapred-site.xml for name=mapred.reduce.child.java.opts causes child tasks to fail at launch

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.2
    • Fix Version/s: 1.1.3
    • Component/s: task
    • Labels:
      None
    • Environment:

      Centos 6.2 32Bit, OpenJDK

      Description

      Hi,

      Below is a patch for Hadoop v1.1.2. I'm new to this list, so if I need to write up a JIRA ticket for this, please let me know.

      The defect scenario is that if you enter any white space within values in this file:
      /etc/hadoop/mapred-site.xml

      e.g.: (a white space prior to the -X...)

      <property>
      <name>mapred.reduce.child.java.opts</name>
      <value> -Xmx1G</value>
      </property>

      All of the child jobs fail, and each child gets an error in the stderr log like:

      Could not find the main class: . Program will exit.

      The root cause is obvious in the patch below - the split on the value was done on whitespace, and any preceding whitespace ultimately becomes a zero-length entry on the child jvm command line, causing the jvm to think that a '' argument is the main class. The patch just skips over any zero-length entries prior to adding them to the jvm vargs list. I looked in trunk as well, to see if the patch would apply there but it looks like Tasks were refactored and this code file is not present any more.

      This error occurred on Open JDK, Centos 6.2, 32 bit.

      Regards,
      Karl

      Index: src/mapred/org/apache/hadoop/mapred/TaskRunner.java
      ===================================================================
      — src/mapred/org/apache/hadoop/mapred/TaskRunner.java (revision 1482686)
      +++ src/mapred/org/apache/hadoop/mapred/TaskRunner.java (working copy)
      @@ -437,7 +437,9 @@
      vargs.add("-Djava.library.path=" + libraryPath);
      }
      for (int i = 0; i < javaOptsSplit.length; i++) {

      • vargs.add(javaOptsSplit[i]);
        + if( javaOptsSplit[i].trim().length() > 0 ) { + vargs.add(javaOptsSplit[i]); + }

        }

      Path childTmpDir = createChildTmpDir(workDir, conf, false);

        Activity

        Show
        Karl D. Gierach added a comment - The patch also should be applied to this file under the current trunk (as noted by Chris Nauroth). https://github.com/apache/hadoop-common/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapred/MapReduceChildJVM.java#L156

          People

          • Assignee:
            Unassigned
            Reporter:
            Karl D. Gierach
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development