Running with the DefaultContainerExecutor it appears that the usercache/<user>/appcache/<appid> directory itself is not removed when the app finishes. All the directories under it are properly removed though.
The nodemanager log file indicates that it tries to delete it:
11/09/23 15:17:56 INFO nodemanager.DefaultContainerExecutor: Deleting absolute path : /home/hadoop/mapred/tmp/mapred-local/usercache/tgraves/appcache/application_1316722920862_0003
This doesn't appear to happen with the LinuxContainerExecutor.