Ran into an issue while running test against faulty data-node code.
Currently in DataNode.java:
This code is called by the DN coordinating the block recovery. In the above case, it is possible for none of the rState (reported by DNs with copies of the replica being recovered) to match the bestState. This can either be caused by faulty DN code or stale/modified/corrupted files on DN. When this happens the DN will end up reporting the minLengh of Long.MAX_VALUE.
Unfortunately there is no check on the NN for replica length. See FSNamesystem.java:
After this point the block length becomes Long.MAX_VALUE. Any subsequent block report (even with correct length) will cause the block to be marked as corrupted. Since this is block could be the last block of the file. If this happens and the client goes away, NN won’t be able to recover the lease and close the file because the last block is under-replicated.
I believe we need to have a sanity check for block size on both DN and NN to prevent such case from happening.