Thanks for reviewing Xiao, new rev attached. Addressed except as per below:
– The new listStatus should return new DirectoryEntries(listing, token, false);
I think it's correct as is. This default implementation lists the entire directory as a single batch, so there is no next batch.
We could do some more work to truncate the listing to start at the passed token and set the returned token to the last element, but I think this falls outside of the spec since hasMore is always false and the token is supposed to come from a previous call and be opaque.
Maybe the helper methods can go to HttpFSUtils?
I think you're talking about toFileStatus here? That method is unfortunately not static since makeQualified is not static, so we can't move it.
Maybe we could reorganize the code in listStatus to parse remainingEntries first, then newToken, so that we don't need 2 duplicate comments.
I just deleted the dup comment. I like ordering the parsing in the order of the arguments to DirectoryEntries.
We'll need (a bunch of) documentation.
Any recommendations on additional docs? The batched listing API is already covered in WebHDFS.md, and I think the javadoc coverage is pretty good.