Thank you for the +1's, Sean Mackrory, Lei (Eddy) Xu and Steve Loughran.
Chris Nauroth, thank you for looking at this patch.
I have added @Ignore annotations where appropriate in version 9. I also removed the copyright lines featuring the year.
Regarding the modification-times of directories in S3A: as directories are "fakes" in s3a, there is no feasible way to get accurate directory timestamps without extensive locking (and coping with slow listings), which counters the rationale of object stores. Therefore, we chose a "dummy" implementation that doesn't break (too many) things.
Setting a fixed time (e.g. epoch) breaks the history server as it looks at the modification time of the directory-object before moving it, and decides the files don't need to be copied if they are "too old". Setting the modificationtime of directories in S3A to System.currentTimeMillis() ensures the historyserver never labels them as being "too old".
Good that you have taken a deeper look into whether always labelling directories as "too young" can give rise to problems in YARN. Looking deeper into the classes LocalResource and LocalResourceType learns that the YARN resource localization is always executed against regular files or .jar-archives (these are the only possible values of LocalResourceType), for which S3A returns the correct timestamps.
However, looking at the AggregatedLogDeletionService of YARN learns that this service will omit removing the appropriate logfiles because the directory will be labelled "too young". I did not find any other places in YARN where this patch can cause problems. I indicated this behaviour in the index.md file in the patch. As this is no breaking situation, I still propose to go forward with this patch.