First off, I think this is a critical feature for HDFS, and I would like to thank all of you folks working on this. In order to make snapshots usable to end users, there is more work to be done in the application software above HDFS and in the management software. So, getting this feature into HDFS expeditiously is very important.
Having read the design by Suresh, Nicholas, Sanjay et. al, and the (counter?) proposal by Aaron, Colin, Todd and Eli - it occurs to me that both designs share a great deal, and that there are really clever ideas in both proposals. Acutely aware of the political sensitivity of this JIRA, I want to say only one thing, hopefully, without offending any of the parties: Considering the amount of work the layers above have to do, it may be in the best interest of the Hadoop community to get this feature into HDFS quickly. Given that Suresh, Nicholas et. al. seem to have working code, and seem much further ahead, it would serve the HDFS community well if all of us could put our weight behind that proposal.
And now, on to my real feedback:
1. Having the ability to snapshot sub-directories of the file system, and to have different snapshot policies for each 'snapshottable' directory is absolutely essential.
2. In your design document, you state that the administrator needs to configure a directory as being snapshottable. It would be useful to allow users to be able to set directories in their own home dir as 'snapshottable'. For example, the administrator provisions a new user xyzzy and his home directory /user/xyzzy. xyzzy may then want to create a dir /user/xyzzy/hbase and want to set it as snapshottable. xyzzy should not have to call the administrator to make his /home/xyzzy/hbase directory snapshottable. I believe that this was discussed in the meetup of last week in terms of an ACL. I think this is important.
Finally, a question:
I still don't understand the 'single client consistency' versus 'multiple client consistency' issues discussed above. Regardless of how many HDFS client connections are active on the namenode, a snapshot will capture the HDFS state at that PIT, right?