Page 2, Concurrency, you mention "mkdir/mkdirs is atomic"
It seems reasonable that mkdir is atomic.
I've been researching mkdirs(), with a focus on idempotence and atomicity.
ClientProtocol.java:mkdirs() clearly labels it as @Idempotent, and the documentation and various implementations support that claim. It's also a property that is relatively straight-forward to implement on many back-end filesystems.
I'm having more difficulty tracking down the atomicity of mkdirs(). The LocalFS implementations are not themselves atomic. I tracked the HDFS implementation back to FSNamesystem.java:mkdirsInt(), which appears to provide an atomic implementation. However, the atomic nature of mkdirsInt() appears to come from
HDFS-988, which looks to fix a bug by making mkdirs() atomic rather having an explicit purpose of making mkdirs() atomic by design.
How are you getting to mkdirs() as atomic?
A mild concern of mine is that even if mkdirs() isn't atomic by design, for HDFS it has been implemented as atomic and who knows who may silently be relying on the not-by-design atomic property. That said, given mkdirs() is idempotent it isn't suitable for use as a locking mechanism.