@Tsuyoshi; I'd spoken with Vinod and others about this a while ago. Should have posted this earlier.. Adding the functionality to the AM in the current state is possible - but will further complicate some components which are already quite complicated - and tough to change.
The TaskAttempt state machine is currently really a mix of TaskAttempt transitions as well as Container transitions. The RMContaienrAllocator is also dealing with more than it should - Nodes, Containers as well as scheduling.
The idea was to split the functionality into a separate TaskAttempt, Container and Node state machine, along with reduced functionality in the scheduler (also decoupling the RM request and AM scheduling). This would make the code cleaner and make re-use (as well as other improvements like handling retired nodes) easier to implement.
Had worked with Vinod on the state transitions, and have been working on the implementation in bits and pieces to see how feasible it is. The code is at https://github.com/sidseth/h2-container-reuse . It's a little bit of a mess at the moment, with lots of TODOs, etc splattered all over, but is just about functional. There's no explicit re-use scheduling yet - but re-use can be tested by running a job which requires more containers than available on the cluster (and some config changes).
the 2nd topic(combining per container) should be moved, because the change seems to be too big.
I believe this was, at least initially, meant to ensure that output from all taskAttempts in one container, would be fetched only once by a reducer (without a common combiner). Either way, that could be a separate jira.