Hadoop Common
  1. Hadoop Common
  2. HADOOP-4953

config property mapred.child.java.opts has maximum length that generates NoClassDefFoundError if exceeded

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.19.0
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
      None
    • Environment:

      Description

      There is an unexpected max length for the value of config property mapred.child.java.opts that, if exceeded, generates an opaque NoClassDefFoundError in child tasks.

      The max length for the value is 146 chars. A length of 147 chars will cause the exception. For example, adding a single extra space between options will convert a working jvm opts clause into one that always generates NoClassDefFoundError when tasktrackers exec child tasks.

      As laboriously diagnosed, conf/hadoop-site.xml was used to set the property and runs were done on "Amazon EC2 Ubuntu 8.04 hardy AMI" (Debian version "lenny/sid") using java 1.6.0_07-b06. Multiple slaves nodes were used and after conf changes, stop-all.sh and start-all.sh were run before each test. The job config props as found on the slave did not appear to have a truncated or damaged value. It made no difference whether @taskid@ appeared at the end or middle of the options and absence of @taskid@ did not eliminate the problem.

      This bug wastes considerable time because the error looks like a classpath problem and even after the java opts property is suspected, a character quoting or unsupported option seems more likely than a length limit.

        Activity

        Paul Baclace created issue -
        Paul Baclace made changes -
        Field Original Value New Value
        Environment Amazon EC2 Ubuntu 8.04 hardy AMI" (Debian version "lenny/sid")
        JDK 1.6.0_07-b06 from Sun
        kernel.ostype = Linux
        kernel.osrelease = 2.6.21.7-2.fc8xen
        kernel.version = #1 SMP Fri Feb 15 12:39:36 EST 2008
        powernow-k8: Found 1 Dual-Core AMD Opteron(tm) Processor 2218 HE processors (version 2.00.00)
        Allen Wittenauer made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Won't Fix [ 2 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Paul Baclace
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development