I see, thanks Haibo Chen.
So with downstream applications like hraven which are used for chargeback on clusters, the accounting for a job is done on the basis of the memory (or vcores) reserved for that job.
Could we wait till I think about this a bit more before reverting from 2.9?
As in your example, my thinking is, if YARN reserves 2GB for a container, then 2GB should be accounted towards that job. If YARN reserves 1Gb, then 1 GB goes towards that job. If this is still true after your patch is applied, then we can keep it. We need to account for whatever memory is actually reserved for a job, regardless of what the job actually uses, since that reserved memory is no longer available for other jobs in that queue/cluster.