[HDFS-8959] Provide an iterator-based API for listing all the snapshottable directories

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: hdfs-client
• Assignee: Rakesh Radhakrishnan
• Reporter: Rakesh Radhakrishnan
• Created: 26/Aug/15 13:00
• Updated: 07/Jan/17 00:57

Description

Presently DistributedFileSystem#getSnapshottableDirListing() is sending all the SnapshottableDirectoryStatus[] array to the clients. Now the client should have enough space to hold it in memory. There could be chance that the client JVMs running out of memory because of this. Also, some time back there was a comment about RPC packet limitation and a large number of snapshot list can again cause issues.

I believe iterator based DistributedFileSystem#listSnapshottableDirs() API would be a good addition!