Redhat EL 5.1, Java 6
While I am coming up a fix for CHUKWA-446, I notice the load average increased by ExecAdaptor. There is probably something that could be improved to reduce the cost of ExecAdaptor.
I just committed this.
Not sure this entirely will resolve issue; I had thought most of the time the ExecAdaptor is blocked waiting on the underlying process, but that may depend which processes are being invoked. Certainly the fix is worthwhile regardless.
Looks like a good catch. I'll commit this.
Retain change in code instead of interface.
Changed "period" unit from millisecond to second. Only converts to millisecond for scheduling.
The expected period parameter is in seconds, but the code is running as:
timer.schedule(execTimer, 0L, period);
Hence, it's running 1000 times more frequent than expected.
The commands are identical before and after. Here is what is in my initial_adaptors:
add org.apache.hadoop.chukwa.datacollection.adaptor.ExecAdaptor Iostat 60 /usr/bin/iostat -x -k 55 2 0
add org.apache.hadoop.chukwa.datacollection.adaptor.ExecAdaptor Df 60 /bin/df -l 0
add org.apache.hadoop.chukwa.datacollection.adaptor.ExecAdaptor Sar 60 /usr/bin/sar -q -r -n ALL 55 0
add org.apache.hadoop.chukwa.datacollection.adaptor.ExecAdaptor Top 60 /usr/bin/top -b -n 1 -c 0
Can you say anything about the configuration settings in use? Which processes are you forking? Same as before, or different?
NM, didn't see graph. Hrm. Let me do some profiling here. But this probably does add more impetus to CHUKWA-419 or equivalent.
Interesting. And sort of surprising, since both are spawning processes in a loop. Increased 50% is how large an increase in absolute terms?
The load one minute average graph for a week. The change was implemented at Jan 16 midnight, and notice the load jumped by 50%.