[ZOOKEEPER-2260] Paginated getChildren call

• Type: New Feature
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.6, 3.5.0
• Labels: api features pull-request-available
• Assignee: Marco P.
• Reporter: Marco P.
• Created: 28/Aug/15 00:29
• Updated: 15/Jun/23 21:09

Description

Add pagination support to the getChildren() call, allowing clients to iterate over children N at the time.

Motivations for this include:

• Getting out of a situation where so many children were created that listing them exceeded the network buffer sizes (making it impossible to recover by deleting)[1]
• More efficient traversal of nodes with large number of children [2]

I do have a patch (for 3.4.6) we've been using successfully for a while, but I suspect much more work is needed for this to be accepted.

[1] https://issues.apache.org/jira/browse/ZOOKEEPER-272
[2] https://issues.apache.org/jira/browse/ZOOKEEPER-282

Issue Links

• blocks: HBASE-14938 Limit the number of znodes for ZK in bulk loaded hfile replication
• is duplicated by: ZOOKEEPER-4383 GetChildren Pagination Feature
• is duplicated by: ZOOKEEPER-282 the getchildren api in zookeeper should return an iterator.
• relates to: ZOOKEEPER-4383 GetChildren Pagination Feature
• relates to: ZOOKEEPER-1162 consistent handling of jute.maxbuffer when attempting to read large zk "directories"
• links to: GitHub Pull Request #50
• links to: GitHub Pull Request #50