Incorporated review comments and synchronized with trunk.
TaskTracker's private method findFreeVirtualMemory should be synchronized on runningTasks.
Done. Synchronized it on the tasktracker object.
Would be good if the resources in TaskTrackerStatus are of the form <String, Writable> instead of <String, Long>.
I was not able to see how to make it work with Writables. The problem is with reading back the objects. It seemed like we need to know what Writable the object actually is, in order to read it back fully. That would have prevented us from making it generic. There is an ObjectWritable, but that seems an overkill.
What I've done instead is to create a small wrapper object that implements Writable called ResourceStatus. This is used to encapsulate all fields that need to be reported to the JT in the TaskTrackerStatus. Please comment if this makes sense.
Just seen that
HADOOP-657 is committed. Do we want to put available disk space also as part of the same resource map?
Modified the availableSpace as a field in ResourceStatus. Ari, can you please check if this change looks OK ? I didn't see a test case in
HADOOP-657 which I could run.
In an offline discussion with Devaraj, he suggested that I not modify the job conf when localizing the task.
Done this. Rather than localizing, I calculate the default free memory when required - for e.g in findFreeVirtualMemory.