Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-4953

config property mapred.child.java.opts has maximum length that generates NoClassDefFoundError if exceeded


    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.19.0
    • Fix Version/s: None
    • Component/s: conf
    • Labels:
    • Environment:


      There is an unexpected max length for the value of config property mapred.child.java.opts that, if exceeded, generates an opaque NoClassDefFoundError in child tasks.

      The max length for the value is 146 chars. A length of 147 chars will cause the exception. For example, adding a single extra space between options will convert a working jvm opts clause into one that always generates NoClassDefFoundError when tasktrackers exec child tasks.

      As laboriously diagnosed, conf/hadoop-site.xml was used to set the property and runs were done on "Amazon EC2 Ubuntu 8.04 hardy AMI" (Debian version "lenny/sid") using java 1.6.0_07-b06. Multiple slaves nodes were used and after conf changes, stop-all.sh and start-all.sh were run before each test. The job config props as found on the slave did not appear to have a truncated or damaged value. It made no difference whether @taskid@ appeared at the end or middle of the options and absence of @taskid@ did not eliminate the problem.

      This bug wastes considerable time because the error looks like a classpath problem and even after the java opts property is suspected, a character quoting or unsupported option seems more likely than a length limit.




            • Assignee:
              pbaclace Paul Baclace
            • Votes:
              0 Vote for this issue
              2 Start watching this issue


              • Created: