TreeMaps are memory inefficient, so I'd prefer to use something else unless we need ordering. I believe ConcurrentHashMap has safe concurrent iteration behavior.
A balanced tree should take up effectively no memory when it's empty, which this tree will be almost all of the time. In contrast, a hash table needs to pre-reserve a chunk of memory to avoid collisions. Minimum capacity for java.util.HashMap seems to be 16, which means you'll have to spend 16 * sizeof(pointer to jobject) = 128 bytes just to do nothing (which is what you'll be doing most of the time). Also, we do need ordering here. Finally, iterating over hash maps is slow because you have to check all the null entries.
A hash map would be a really terrible thing to use here, because if it did ever grow, that would be just a transitory event. And then it would never shrink again, but just continue using that memory forever. Check HashMap.java if you are curious.
Any reason for the StreamInfo class over a Long?
StreamInfo is mutable; Long is not. Although I could remove and re-insert the element into the map, it seems unnecessary.
Unrelated to this patch, but DFSClient#endFileLease is poorly named, since it doesn't actually end the lease. Maybe rename it to stopLeaseRenewal?
OK, I'll change it to stopLeaseRenewal.
TestHdfsClose: should use GenericTestUtils#assertExceptionContains when catching an expected IOException.
It's clearly going to throw some IOException, because the DataNode/NameNode/etc are down, but I don't think the message inside the IOException is important.
I noticed that endFileLease is now called before completeFile, could the lease expire between these two calls? I guess it'll still get cleaned up okay by the ZSM, but this is extra work.
I suppose I might as well wait until after completeFile to end the lease renewal. As you've already inferred, it's extremely unlikely to make a difference, since if there are absolutely any other files open by the client, the lease will still be renewed.