Ok! Here's the skeleton code that has come out of my attempt to add this functionality to the NameNode. DISCLAIMER: This patch is not ready and I'm uploading it only so that you folks can see what I'm thinking so far.
I would request feedback on the following (and whatever else you think of):
1. Should HdfsUsageMetricsSource be thread safe? Should I just assume the FSN write lock is always held when calling into here?
2. I understand that we need to plug into a LOT of places to correctly update the stats. I have only plugged into 2-3 places (so obviously the usage will be incorrect if you venture out of those ops: create / delete / chown files+dirs and even these have wrinkles I need to smooth) . I propose we do this all as another sub-task after the framework gets committed.
3. I still need to figure out how best to let this be configurable for any of the HDFS daemons: NameNode/Standby/SecondaryNamenode
4. Enable and disable this feature dynamically.