Resolution: Won't Fix
Affects Version/s: 0.19.0
Fix Version/s: None
Amazon EC2 Ubuntu 8.04 hardy AMI" (Debian version "lenny/sid")
JDK 1.6.0_07-b06 from Sun
kernel.ostype = Linux
kernel.osrelease = 18.104.22.168-2.fc8xen
kernel.version = #1 SMP Fri Feb 15 12:39:36 EST 2008
powernow-k8: Found 1 Dual-Core AMD Opteron(tm) Processor 2218 HE processors (version 2.00.00)
There is an unexpected max length for the value of config property mapred.child.java.opts that, if exceeded, generates an opaque NoClassDefFoundError in child tasks.
The max length for the value is 146 chars. A length of 147 chars will cause the exception. For example, adding a single extra space between options will convert a working jvm opts clause into one that always generates NoClassDefFoundError when tasktrackers exec child tasks.
As laboriously diagnosed, conf/hadoop-site.xml was used to set the property and runs were done on "Amazon EC2 Ubuntu 8.04 hardy AMI" (Debian version "lenny/sid") using java 1.6.0_07-b06. Multiple slaves nodes were used and after conf changes, stop-all.sh and start-all.sh were run before each test. The job config props as found on the slave did not appear to have a truncated or damaged value. It made no difference whether @taskid@ appeared at the end or middle of the options and absence of @taskid@ did not eliminate the problem.
This bug wastes considerable time because the error looks like a classpath problem and even after the java opts property is suspected, a character quoting or unsupported option seems more likely than a length limit.